Pith. sign in

REVIEW 4 major objections 6 minor 53 references

Orchestration for Domain-specific Edge-Cloud Language Models

T0 review · 4 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Tuning the whole LLM pipeline per query beats model routing alone, cutting cost 60% and latency 6x.

desk verdict ECO-LLM makes a real case that per-query joint optimization across the serving pipeline beats model-routing alone, but the evaluation overclaims SLO compliance and leans entirely on LLM judges; worth refereeing with revisions. read the letter →

arxiv 2507.09003 v2 pith:HXHE227V submitted 2025-07-11 cs.DB

Prasoon Patidar (1) , Alex Crown (2) , Kevin Hsieh (2) , Yifei Xu (2) , Tusher Chakraborty (2) , Ranveer Chandra (2) , Yuvraj Agarwal (1) ((1) Carnegie Mellon University , (2) Microsoft Research) This is my paper
classification cs.DB
keywords edge-cloudorchestrationLLMservingpipelinequery-leveloptimizationmodelroutingretrieval-augmentedgenerationSLO-awareselectiondomainadaptationLLM-as-a-judge
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ECO-LLM is a system for deploying domain-specific edge-cloud assistants that treats the serving pipeline—query preprocessing, context retrieval, context processing, and model selection—as one joint optimization problem, solved per query. The paper's central claim is that this joint orchestration consistently beats model-routing-only systems such as RouteLLM: costs drop by roughly 60%, latency improves by up to 6x, and accuracy stays in a stable 73-87% band across five domains, whereas routing-only accuracy swings from 54% to 85%. The strongest evidence is exhaustive path selection, the Oracle, averaging 94% accuracy versus 85% for GPT-4.1, showing that component interplay matters beyond model choice. The system makes domain adaptation practical by exploring the configuration space with clustering and pareto-optimal sampling rather than exhaustive evaluation, then choosing paths at runtime under user-defined cost and latency SLOs.

What carries the argument

The load-bearing object is the query resolution path, a tuple $P = ((q,\theta_q),(r,\theta_r),(c,\theta_c),(m,\theta_m))$ that fixes the implementation and configuration of four pipeline modules: query processing, retrieval, context processing, and model. The argument advances through three mechanisms: Critical Component Analysis computes per-query impact scores to identify which components must stay fixed, Domain-Specific Query Encoding learns prototype vectors and a projection network that map a query's embedding to the component set it needs even when surface-semantic similarity is misleading, and Runtime Path Selection filters all paths by SLO constraints and critical components, then scores the survivors against nearest training queries. The Emulator's stratified budget allocation cuts exploration from $O(|Q|\times|P|)$ to $O(\sqrt{|Q|}|P| + |Q|\sqrt{|P|})$ while staying within a few accuracy points of exhaustive exploration.

What would settle it

Run the same five domains with a human expert rating on a random sample of responses, or a third judge from a different model family, and check whether ECO-LLM's accuracy advantage over RouteLLM holds; an accuracy rank reversal or a large judge disagreement would undercut the central claim.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that the optimal way to resolve an LLM query is not to pick the best model but to pick an entire query resolution path: an ordered tuple $P = ((q,\theta_q),(r,\theta_r),(c,\theta_c),(m,\theta_m))$ of query processing, retrieval strategy, context processing, and model, with each module carrying its own implementation options and parameters. The ECO-LLM Emulator characterizes these paths for a target domain and hardware setup using stratified budget allocation and critical-component analysis, and the Runtime encodes unseen queries into a domain-specific space, filters paths to those that contain the critical components and satisfy SLO constraints, and scores the survivors by nearest-neighbor evidence. Across automotive, smart home, agriculture, technical support, and IoT security domains on M4-class hardware, the system matches or beats cloud-only GPT-4.1 and model-routing baselines on cost and latency while keeping accuracy stable, and the Oracle result (94% vs 85% average accuracy over GPT-4.1) indicates that joint component selection itself, not model capability, drives much of the gain.

Load-bearing premise

The evaluation's accuracy numbers come from LLM judges (GPT-4o and Gemini-2.5-Flash) treated as ground truth, so if those judges favor particular response styles or model families, every accuracy comparison and SLO-accuracy plot could shift.

Editorial extensions

If this is right

  • Domain-specific edge assistants can be deployed from documentation alone: the Context Generator produces training queries, and the Emulator profiles paths without manual tuning.
  • Model-routing-only systems will keep degrading on domains where queries need coordinated preprocessing, so practitioners should budget for joint selection rather than stronger routers alone.
  • M4-class edge devices (30-40 TOPS, 20-32GB RAM) are a practical target for responsive assistants, while weaker devices produce prohibitive latencies and stronger ones give diminishing returns.
  • SLO violations that persist even at relaxed constraints become a signal that the domain's constraints are infeasible, instead of being hidden by silent accuracy loss.
  • Reduced-budget exploration, around 35-70% of queries, is enough for near-equivalent deployment quality on most domains.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same joint-optimization framing should extend to pipeline choices not implemented here, such as prompt templates, tool selection, or retriever ensembling, since the module-manager interfaces are already extensible.
  • The observation that constrained exploration sometimes outperforms exhaustive exploration suggests exhaustive search can overfit training queries; a direct test would measure held-out generalization across multiple random query splits.
  • The Oracle-vs-GPT-4.1 gap implies component synergy contributes roughly nine accuracy points beyond model choice; decomposing that gap by toggling one module at a time would quantify each component's share.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes ECO-LLM, a two-stage framework for edge-cloud LLM serving. The Emulator enumerates and explores a combinatorial space of pipeline configurations—query processing, retrieval, context processing, and model selection—using a stratified budget allocation, collecting per-query accuracy, latency, and cost metrics. The Runtime uses Critical Component Analysis, a learned Domain-Specific Query Encoding, and constrained path scoring to select a path per query under user-specified latency and cost SLOs. The evaluation compares ECO-LLM against RouteLLM and GPT-4.1 on five domains and four hardware platforms, reporting substantially lower cost and latency at comparable accuracy, and claiming near-zero SLO violations when constraints are feasible.

Significance. The central idea—treating component selection across the whole serving pipeline rather than model routing alone—is timely and potentially useful for domain-specific edge deployments. The system is engineered with modular managers, prefix caching, and an explicit problem formulation (Eqs. 1–6), and the authors are honest about deployment limitations in Section 6. If the accuracy measurements were validated by human judgment and the SLO claims were properly qualified, the framework could be a practical contribution. In its current form, however, the quantitative claims rest on LLM-judged synthetic labels and single-point comparisons, and the SLO attainment claim is overstated relative to the paper's own reported violation rates.

major comments (4)
  1. [Abstract, Section 5.5, Figure 4] The abstract claims ECO-LLM is "consistently adhering to specified latency and cost constraints," and the conclusion claims it meets user-defined SLOs, but Section 5.5 and Figure 4 report persistent violation rates of 15–25% for SmartHome cost SLOs even at $10 and 15–20% for TechQA at higher cost constraints. The paper explains these as a design choice to prioritize accuracy over compliance, but that explanation changes the claim rather than supporting it. Please revise the headline claim to "meets SLOs when feasible," state the observed violation rates as a function of constraint tightness, and avoid unqualified statements in the abstract and conclusion.
  2. [Section 4, Section 5.1, Section 3.2.3] The accuracy metric is G-Eval with two LLM judges (GPT-4o and Gemini-2.5-Flash) comparing generated responses against ground-truth answers. For the automotive and smart home domains, the queries and reference answers were themselves generated by ECO-LLM's Context Generator (Section 3.2.3), so the reported accuracy measures agreement with LLM-generated references on LLM-generated queries, not correctness against human judgment. Since these labels are used to train CCA/DSQE and to score every accuracy table (Tables 3–5), the central accuracy comparisons, the Oracle-vs-GPT-4.1 argument in Table 4, and the ablations could all be biased by judge or reference style preferences. Please add a human evaluation on a sample of responses across all five domains, report judge–human agreement, and show that the main comparisons are robust to that validation.
  3. [Tables 3, 4, and 5; Sections 5.2–5.4] All headline comparisons are reported as single-point accuracy, cost, and latency values with no confidence intervals, error bars, or significance tests. Figure 4 provides 99% confidence intervals for SLO violation curves, but the main tables do not. Without variance estimates it is difficult to assess whether the claimed 60% cost reduction, 6× latency improvement, and "consistent 73–87% accuracy" are robust or within noise. Please add repeated-run or bootstrap confidence intervals, at least for the M4 platform, and perform significance tests for the main ECO-LLM versus RouteLLM comparisons.
  4. [Section 5.6, Table 6] The claim that stratified exploration achieves "near-equivalent performance with up to 65% fewer evaluations" is not supported by Table 6 at the 35% exploration level: IoT Security Cost-First degrades by 7.2 percentage points and Smart Home Latency-First by 4.5 percentage points relative to full exploration. The "within 1.5 points" statement holds only at 70% exploration. Please restrict the claim to the 70% budget or provide an explicit justification for accepting multi-point accuracy drops on selected domains in exchange for a 65% reduction in evaluations.
minor comments (6)
  1. [Abstract] The abstract contains two different sets of headline numbers for the same system: "90% vs. 74% accuracy, 90% cost reduction, 55% latency reduction" for exhaustive exploration and "60% cost reduction, up to 6× latency, 73–87% accuracy" for practical deployment. Please label these comparison scenarios explicitly to avoid confusion.
  2. [Section 3.3.3 and Section 4] Section 4 lists LLMLingua under query processing, but the example critical component set in Section 3.3.3 places LLMLingua under θc (context processing). Please align the notation so the component type is consistent.
  3. [Section 5.1] RouteLLM is described as "augmented with our RAG module for fair comparison," but no details are given for this augmentation. Specify the retrieval configuration and how it was selected; otherwise the baseline comparison is not fully reproducible.
  4. [Section 5.2] There are typos in this section: "deplopying" should be "deploying," and "cable edge device" should be "capable edge device."
  5. [Tables 3 and 4] The color legend (green/blue/mauve) is not visible in monochrome print; please add symbols or textual markers so the best values are identifiable in grayscale.
  6. [Figure 2] The caption states that marker size represents response quality, but the figure is not legible at the current resolution. Please enlarge the figure or tabulate the underlying data.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: ECO-LLM's central claims are supported by measured generalization to unseen queries and external baselines, not by definitional reduction.

full rationale

Walking the derivation chain from problem formulation (Section 3.1), CCA (Algorithm 2), DSQE (Section 3.3.3), RPS (Algorithm 3), to evaluation (Section 5), no load-bearing step reduces by construction to its inputs. The same G-Eval/LLM-judge accuracy metric (Section 4, Section 5.1) is used both to derive CCA labels for training and to score final responses, but this is standard supervised learning: DSQE is trained offline on labeled queries and evaluation measures accuracy on previously unseen queries selected by the runtime, so the reported 60% cost reduction / 6x latency improvement and 73-87% accuracy are measured outcomes, not fitted identities. Path scoring (Equation 14) is a k-NN aggregation over training data, but the actual accuracy of the selected path on a new query is independently measured. The Oracle is an explicit upper bound computed from exhaustive per-query accuracy, not a claimed prediction. Cost is estimated from fixed OpenAI pricing (Equation 3) and SLO constraints are user inputs (Equation 4), so SLO attainment is not definitionally guaranteed. There are no load-bearing self-citations and no uniqueness theorem imported from the authors' prior work. The paper honestly discloses limitations: Section 5.5 reports persistent cost SLO violations (SmartHome 15-25%, TechQA 15-20%) as a deliberate accuracy-first design choice, and Section 6 notes DSQE degrades on out-of-distribution queries and requires retraining. These are validity/consistency concerns about the abstract's 'consistently adhering' wording; they do not constitute circular derivation. Therefore the paper is self-contained against external benchmarks (RouteLLM, TechQA, AgriQA) and merits a circularity score of 0.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical entities. The free parameters are mostly exploration and threshold knobs, plus unspecified DSQE hyperparameters. The key domain assumptions are the LLM judge validity, the linear cost model, and the representativeness of generated queries.

free parameters (4)
  • Budget factor B = 2, 5, 10
    Controls how many representative queries and paths are explored in stratified budget allocation; introduced in Section 3.2.4 Algorithm 1 and varied in Section 5.6.
  • Accuracy threshold tau = not specified
    Used in Critical Component Analysis (Algorithm 2) to decide whether a component's impact is critical; no value or tuning procedure is given.
  • Optimization preference lambda = 0 or 1
    Selects cost-first versus latency-first tie-breaking in CCA and runtime decisions; defined in Section 3.3.2.
  • DSQE loss weights alpha and beta = not specified
    Weights for diversity and regularization terms in the DSQE training objective (Equation 12); no values are reported.
assumptions (3)
  • domain assumption LLM-as-a-judge (G-Eval with GPT-4o and Gemini-2.5-Flash) provides a valid and unbiased accuracy metric for all five domains.
    Invoked in Section 4 and Section 5.1 as the response quality measure. If the judges are biased, the central accuracy comparisons and ablations are unreliable.
  • domain assumption Cost is a linear function of input and output token counts with per-model pricing constants (Equation 3).
    Assumes no fixed overheads, no latency-based pricing, and no economies of scale. Used throughout the evaluation for cost reporting.
  • domain assumption The generated queries from the Context Generator for automotive and smart home domains are representative of real user queries in those domains.
    Introduced in Section 3.2.3 and used for evaluation in Section 5.1. If the generated queries are unrepresentative, the cross-domain generalization claims are weakened.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Orchestration for Domain-specific Edge-Cloud Language Models." pith.science (2026). https://pith.science/paper/HXHE227V

@misc{pith2026250709003,
  author       = {Pith},
  title        = {Pith review of: Orchestration for Domain-specific Edge-Cloud Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HXHE227V}},
  note         = {Machine review of arXiv:2507.09003}
}
read the original abstract

The remarkable performance of Large Language Models (LLMs) has inspired many applications, which often necessitate edge-cloud collaboration due to connectivity, privacy, and cost considerations. Traditional methods primarily focus on selecting the best LLM model for optimizing performance, while neglecting the critical interplay between the components of the LLM serving pipeline (context retrieval, query preprocessing, etc.) or the changing latency and cost constraints. We introduce ECO-LLM (Edge-Cloud Orchestrator for LLMs), a novel system that reframes this problem as a joint optimization challenge and solves it by systematically exploring component configurations and dynamically selecting optimal strategies at the query level. ECO-LLM consists of two components: (1) the ECO-LLM Emulator, which efficiently explores the vast configuration space utilizing query clustering and pareto-optimal path selection, gathering domain-specific performance metrics without exhaustive evaluation; and (2) the ECO-LLM Runtime, which leverages these metrics to dynamically select optimal resolution strategies for user queries while meeting user-defined Service Level Objectives (SLOs). We evaluate ECO-LLM on a smart home and a smart car assistant scenarios. With an exhaustive exploration of all possible configurations for seen queries, ECO-LLM outperforms cloud-based models like GPT-4o in terms of accuracy (90% vs. 74% on average) while reducing costs by 90% and latency by 55%, demonstrating the value of its joint optimization at the query level. In practical deployment for previously unseen queries, ECO-LLM selects configurations that reduce costs by 62% or improve response times by 62% on average compared to state-of-the-art model routing approaches, while maintaining higher accuracy and consistently adhering to specified latency and cost constraints.

Figures

Figures reproduced from arXiv: 2507.09003 by the authors.

Figure 2
Figure 2. Performance variations in query resolution paths for [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Overall Architecture of ECO-LLM, showing the components of the [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. SLO attainment across four domains. Figures (a) [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

53 extracted references · 23 canonical work pages

  1. [1]

    Automix: Automatically mixing language models.arXiv preprint arXiv:2310.12963, 2023

    Pranjal Aggarwal, Aman Madaan, Ankit Anand, Sriv- idya Pranavi Potharaju, Swaroop Mishra, Pei Zhou, Aditya Gupta, Dheeraj Rajagopal, Karthik Kappaganthu, Yiming Yang, et al. Automix: Automatically mixing language models.arXiv preprint arXiv:2310.12963, 2023

    arXiv

  2. [2]

    Deepeval: The llm evaluation framework

    Confident AI. Deepeval: The llm evaluation framework. https://github.com/confident-ai/ deepeval, 2024

    work page 2024

  3. [3]

    SmolLM2: When Smol goes big – data-centric training of a small language model, 2025

    Loubna Ben Allal, Anton Lozhkov, Elie Bakouch, Gabriel Martín Blázquez, Guilherme Penedo, Lewis Tunstall, Andrés Marafioti, Hynek Kydlíˇcek, Agustín Pi- queres Lajarín, Vaibhav Srivastav, Joshua Lochner, Caleb Fahlgren, Xuan-Son Nguyen, Clémentine Four- rier, Ben Burtenshaw, Hugo Larcher, Haojun Zhao, Cyril Zakka, Mathieu Morlon, Colin Raffel, Leandro von...

    work page 2025

  4. [4]

    Packt Publishing Ltd, 2023

    Valentina Alto.Modern Generative AI with ChatGPT and OpenAI Models: Leverage the capabilities of Ope- nAI’s LLM for productivity and innovation with GPT3 and GPT4. Packt Publishing Ltd, 2023

    work page 2023

  5. [5]

    Improving rag quality for large language models with topic-enhanced reranking

    Nicholas Ampazis. Improving rag quality for large language models with topic-enhanced reranking. In IFIP International Conference on Artificial Intelligence Applications and Innovations, pages 74–87. Springer, 2024

    work page 2024

  6. [6]

    Apple intelligence

    Apple. Apple intelligence. https://www.apple.com/ apple-intelligence/, Retrieved on 2024-11

    work page 2024

  7. [7]

    Agxqa: Agricultural question answer- ing dataset

    MSU CECO. Agxqa: Agricultural question answer- ing dataset. https://huggingface.co/datasets/ msu-ceco/agxqa_v1, 2024

    work page 2024

  8. [8]

    Kwok, and Yu Zhang

    Shuhao Chen, Weisen Jiang, Baijiong Lin, James T. Kwok, and Yu Zhang. RouterDC: Query-Based Router by Dual Contrastive Learning for Assembling Large Language Models, September 2024. arXiv:2409.19886 [cs]

    arXiv 2024

Show all 53 references
  1. [9]

    A unified approach to routing and cascading for llms.arXiv preprint arXiv:2410.10347, 2024

    Jasper Dekoninck, Maximilian Baader, and Martin Vechev. A unified approach to routing and cascading for llms.arXiv preprint arXiv:2410.10347, 2024

    2024 arXiv

  2. [10]

    Hybrid llm: Cost-efficient and quality-aware query routing.arXiv preprint arXiv:2404.14618, 2024

    Dujian Ding, Ankur Mallick, Chi Wang, Robert Sim, Subhabrata Mukherjee, Victor Ruhle, Laks VS Laksh- manan, and Ahmed Hassan Awadallah. Hybrid llm: Cost-efficient and quality-aware query routing.arXiv preprint arXiv:2404.14618, 2024

    2024 arXiv

  3. [11]

    The llama 3 herd of models

    Aaron Grattafiori et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  4. [12]

    Gemma: Open models based on gemini research and technology.arXiv preprint arXiv:2403.08295, 2024

    Gemma Team et al. Gemma: Open models based on gemini research and technology.arXiv preprint arXiv:2403.08295, 2024

    2024 arXiv

  5. [13]

    Phi-3 technical report: A highly capable language model locally on your phone.arXiv preprint arXiv:2404.14219, 2024

    Marah Abdin et al. Phi-3 technical report: A highly capable language model locally on your phone.arXiv preprint arXiv:2404.14219, 2024

    2024 arXiv

  6. [14]

    Autorag-hp: Automatic online hyper-parameter tuning for retrieval-augmented generation.arXiv preprint arXiv:2406.19251, 2024

    Jia Fu, Xiaoting Qin, Fangkai Yang, Lu Wang, Jue Zhang, Qingwei Lin, Yubo Chen, Dongmei Zhang, Sar- avan Rajmohan, and Qi Zhang. Autorag-hp: Automatic online hyper-parameter tuning for retrieval-augmented generation.arXiv preprint arXiv:2406.19251, 2024

    2024 arXiv

  7. [15]

    Retrieval-augmented generation for large language models: A survey.arXiv preprint arXiv:2312.10997, 2024

    Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang. Retrieval-augmented generation for large language models: A survey.arXiv preprint arXiv:2312.10997, 2024

    2024 arXiv

  8. [16]

    Gemini makes your mobile device a powerful AI assistant

    Google. Gemini makes your mobile device a powerful AI assistant. https://blog.google/products/ gemini/made-by-google-gemini-ai-updates/ , Retrieved on 2024-11

    2024

  9. [17]

    Getting started with azure openai

    Shimon Ifrah. Getting started with azure openai

  10. [18]

    Smart home iot security dataset

    IoTSmart. Smart home iot security dataset. https: //github.com/IoTSmart-art/smarthome, 2024

    2024

  11. [19]

    Longllm- lingua: Accelerating and enhancing llms in long con- text scenarios via prompt compression.arXiv preprint arXiv:2310.06839, 2023

    Huiqiang Jiang, Qianhui Wu, Xufang Luo, Dongsheng Li, Chin-Yew Lin, Yuqing Yang, and Lili Qiu. Longllm- lingua: Accelerating and enhancing llms in long con- text scenarios via prompt compression.arXiv preprint arXiv:2310.06839, 2023

    arXiv

  12. [20]

    CE-CoLLM: Efficient and Adaptive Large Language Models Through Cloud- Edge Collaboration, June 2025

    Hongpeng Jin and Yanzhao Wu. CE-CoLLM: Efficient and Adaptive Large Language Models Through Cloud- Edge Collaboration, June 2025. arXiv:2411.02829 [cs]

    2025 arXiv

  13. [21]

    Evaluation of hypothetical document and query embeddings for information retrieval enhancements in the context of diverse user queries

    Marten Jostmann and Hendrik Winkelmann. Evaluation of hypothetical document and query embeddings for information retrieval enhancements in the context of diverse user queries. 2024

    2024

  14. [22]

    Hypa-rag: A hybrid parameter adaptive retrieval- augmented generation system for ai legal and policy applications.arXiv preprint arXiv:2409.09046, 2024

    Rishi Kalra, Zekun Wu, Ayesha Gulley, Airlie Hilliard, Xin Guan, Adriano Koshiyama, and Philip Treleaven. Hypa-rag: A hybrid parameter adaptive retrieval- augmented generation system for ai legal and policy applications.arXiv preprint arXiv:2409.09046, 2024

    2024 arXiv

  15. [23]

    Venieris, Alexandros Kouris, Rui Li, and Nicholas D

    Stefanos Laskaridis, Stylianos I. Venieris, Alexandros Kouris, Rui Li, and Nicholas D. Lane. The future of consumer Edge-AI computing.IEEE Pervasive Com- put., 23(3):21–30, 2024. 13

    2024

  16. [24]

    Mentor- kd: Making small language models better multi-step reasoners.arXiv preprint arXiv:2410.09037, 2024

    Hojae Lee, Junho Kim, and SangKeun Lee. Mentor- kd: Making small language models better multi-step reasoners.arXiv preprint arXiv:2410.09037, 2024

    2024 arXiv

  17. [25]

    Retrieval-augmented generation for knowledge- intensive nlp tasks.Advances in Neural Information Processing Systems, 33:9459–9474, 2020

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. Retrieval-augmented generation for knowledge- intensive nlp tasks.Advances in Neural Information Processing Systems...

    2020

  18. [26]

    Patrick S. H. Lewis, Ethan Perez, Aleksandra Pik- tus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rock- täschel, Sebastian Riedel, and Douwe Kiela. Retrieval- augmented generation for knowledge-intensive NLP tasks. InAdvances...

    2020

  19. [27]

    G-eval: Nlg evalu- ation using gpt-4 with better human alignment.arXiv preprint arXiv:2303.16634, 2023

    Yang Liu, Dan Iter, Yichong Xu, Shuohang Wang, Ruochen Xu, and Chenguang Zhu. G-eval: Nlg evalu- ation using gpt-4 with better human alignment.arXiv preprint arXiv:2303.16634, 2023

    2023 arXiv

  20. [28]

    Llmproxy: Reducing cost to access large lan- guage models.arXiv preprint arXiv:2410.11857, 2024

    Noah Martin, Abdullah Bin Faisal, Hiba Eltigani, Rukhshan Haroon, Swaminathan Lamelas, and Fahad Dogar. Llmproxy: Reducing cost to access large lan- guage models.arXiv preprint arXiv:2410.11857, 2024

    2024

  21. [29]

    Copilot+ PC

    Microsoft. Copilot+ PC. https://www. microsoft.com/en-us/windows/business/ devices/copilot-plus-pcs, Retrieved on 2024-11

    2024

  22. [30]

    Routoo: Learning to Route to Large Language Models Effectively, October 2024

    Alireza Mohammadshahi, Arshad Rafiq Shaikh, and Majid Yazdani. Routoo: Learning to Route to Large Language Models Effectively, October 2024. arXiv:2401.13979 [cs]

    2024 arXiv

  23. [31]

    Metallm: A high-performant and cost-efficient dynamic framework for wrapping llms.arXiv preprint arXiv:2407.10834, 2024

    Quang H Nguyen, Duy C Hoang, Juliette Decugis, Saurav Manchanda, Nitesh V Chawla, and Khoa D Doan. Metallm: A high-performant and cost-efficient dynamic framework for wrapping llms.arXiv preprint arXiv:2407.10834, 2024

    2024 arXiv

  24. [32]

    Ollama. Ollama. https://github.com/ollama/ ollama, 2024

    2024

  25. [33]

    Routellm: Learning to route llms with preference data.arXiv preprint arXiv:2406.18665, 2024

    Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E Gonzalez, M Waleed Kadous, and Ion Stoica. Routellm: Learning to route llms with preference data.arXiv preprint arXiv:2406.18665, 2024

    2024 arXiv

  26. [34]

    Dynamic LLM Routing and Se- lection based on User Preferences: Balancing Perfor- mance, Cost, and Ethics.International Journal of Computer Applications, 186(51):1–7, November 2024

    Deepak Babu Piskala, Vijay Raajaa, Sachin Mishra, and Bruno Bozza. Dynamic LLM Routing and Se- lection based on User Preferences: Balancing Perfor- mance, Cost, and Ethics.International Journal of Computer Applications, 186(51):1–7, November 2024. arXiv:2502.16696 [cs]

    2024 arXiv

  27. [35]

    Techqa: A dataset for techni- cal question answering

    Ganesh Rajagopalan et al. Techqa: A dataset for techni- cal question answering. https://huggingface.co/ datasets/rojagtap/tech-qa, 2024

    2024

  28. [36]

    Phi-4: Technical report, 2024

    Microsoft Research. Phi-4: Technical report, 2024

    2024

  29. [37]

    All-mpnet at semeval-2024 task 1: Ap- plication of mpnet for evaluating semantic textual relat- edness

    Marco Siino. All-mpnet at semeval-2024 task 1: Ap- plication of mpnet for evaluating semantic textual relat- edness. InProceedings of the 18th International Work- shop on Semantic Evaluation (SemEval-2024), pages 379–384, 2024

    2024

  30. [38]

    Tensoropera router: A multi-model router for efficient llm inference, 2024

    Dimitris Stripelis, Zijian Hu, Jipeng Zhang, Zhaozhuo Xu, Alay Dilipbhai Shah, Han Jin, Yuhang Yao, Salman Avestimehr, and Chaoyang He. Tensoropera router: A multi-model router for efficient llm inference, 2024

    2024

  31. [39]

    Teola: Towards end-to-end optimization of llm-based applica- tions.arXiv preprint arXiv:2407.00326, 2024

    Xin Tan, Yimin Jiang, Yitao Yang, and Hong Xu. Teola: Towards end-to-end optimization of llm-based applica- tions.arXiv preprint arXiv:2407.00326, 2024

    2024

  32. [40]

    Self- preference bias in llm-as-a-judge, 2025

    Koki Wataoka, Tsubasa Takahashi, and Ryokan Ri. Self- preference bias in llm-as-a-judge, 2025

    2025

  33. [41]

    Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems, 35:24824–24837, 2022

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models.Advances in neural information processing systems, 35:24824–24837, 2022

    2022

  34. [42]

    Corrective retrieval augmented generation.arXiv preprint arXiv:2401.15884, 2024

    Shi-Qi Yan, Jia-Chen Gu, Yun Zhu, and Zhen-Hua Ling. Corrective retrieval augmented generation.arXiv preprint arXiv:2401.15884, 2024

    2024 arXiv

  35. [43]

    EdgeFM: Leveraging Foundation Model for Open-set Learning on the Edge

    Bufang Yang, Lixing He, Neiwen Ling, Zhenyu Yan, Guoliang Xing, Xian Shuai, Xiaozhe Ren, and Xin Jiang. EdgeFM: Leveraging Foundation Model for Open-set Learning on the Edge. InProceedings of the 21st ACM Conference on Embedded Networked Sensor Systems, pages 111–124, Istanbul...

  36. [44]

    PerLLM: Personalized In- ference Scheduling with Edge-Cloud Collaboration for Diverse LLM Services, May 2024

    Zheming Yang, Yuanhao Yang, Chang Zhao, Qi Guo, Wenkai He, and Wen Ji. PerLLM: Personalized In- ference Scheduling with Edge-Cloud Collaboration for Diverse LLM Services, May 2024. arXiv:2405.14636 [cs]

    2024 arXiv

  37. [45]

    Scalellm: A resource-frugal llm serving framework by optimizing end-to-end efficiency.arXiv preprint arXiv:2408.00008, 2024

    Yuhang Yao, Han Jin, Alay Dilipbhai Shah, Shanshan Han, Zijian Hu, Yide Ran, Dimitris Stripelis, Zhaozhuo Xu, Salman Avestimehr, and Chaoyang He. Scalellm: A resource-frugal llm serving framework by optimizing end-to-end efficiency.arXiv preprint arXiv:2408.00008, 2024. 14

    2024 arXiv

  38. [46]

    EDGE- LLM: Enabling Efficient Large Language Model Adap- tation on Edge Devices via Layerwise Unified Compres- sion and Adaptive Layer Tuning and V oting, June 2024

    Zhongzhi Yu, Zheng Wang, Yuhan Li, Haoran You, Ruijie Gao, Xiaoya Zhou, Sreenidhi Reedy Bommu, Yang Katie Zhao, and Yingyan Celine Lin. EDGE- LLM: Enabling Efficient Large Language Model Adap- tation on Edge Devices via Layerwise Unified Compres- sion and Adaptive Layer Tuning...

    2024 arXiv

  39. [47]

    Mpmqa: multimodal question answering on product manuals

    Liang Zhang, Anwen Hu, Jing Zhang, Shuo Hu, and Qin Jin. Mpmqa: multimodal question answering on product manuals. InProceedings of the AAAI Conference on Artificial Intelligence, volume 37, pages 13958–13966, 2023

    2023

  40. [48]

    Edgeshard: Efficient llm inference via collaborative edge computing.arXiv preprint arXiv:2405.14371, 2024

    Mingjin Zhang, Jiannong Cao, Xiaoming Shen, and Zeyang Cui. Edgeshard: Efficient llm inference via collaborative edge computing.arXiv preprint arXiv:2405.14371, 2024

    2024 arXiv

  41. [49]

    A survey of large language models.arXiv preprint arXiv:2303.18223, 2024

    Wayne Xin Zhao, Kun Zhou, Junyi Li, Tianyi Tang, Xiaolei Wang, Yupeng Hou, Yingqian Min, Beichen Zhang, Junjie Zhang, Zican Dong, Yifan Du, Chen Yang, Yushuo Chen, Zhipeng Chen, Jinhao Jiang, Ruiyang Ren, Yifan Li, Xinyu Tang, Zikang Liu, Peiyu Liu, Jian- Yun Nie, and Ji-Rong ...

    2024 arXiv

  42. [50]

    Eagle: Ef- ficient training-free router for multi-llm inference.arXiv preprint arXiv:2409.15518, 2024

    Zesen Zhao, Shuowei Jin, and Z Morley Mao. Eagle: Ef- ficient training-free router for multi-llm inference.arXiv preprint arXiv:2409.15518, 2024

    2024 arXiv

  43. [51]

    Take a step back: Evoking reasoning via ab- straction in large language models.arXiv preprint arXiv:2310.06117, 2023

    Huaixiu Steven Zheng, Swaroop Mishra, Xinyun Chen, Heng-Tze Cheng, Ed H Chi, Quoc V Le, and Denny Zhou. Take a step back: Evoking reasoning via ab- straction in large language models.arXiv preprint arXiv:2310.06117, 2023

    2023 arXiv

  44. [52]

    A survey on model compression for large lan- guage models.CoRR, abs/2308.07633, 2023

    Xunyu Zhu, Jian Li, Yong Liu, Can Ma, and Weiping Wang. A survey on model compression for large lan- guage models.CoRR, abs/2308.07633, 2023

    2023 arXiv

  45. [53]

    EmbedLLM: Learning Compact Representations of Large Language Models, October 2024

    Richard Zhuang, Tianhao Wu, Zhaojin Wen, Andrew Li, Jiantao Jiao, and Kannan Ramchandran. EmbedLLM: Learning Compact Representations of Large Language Models, October 2024. arXiv:2410.02223 [cs]. 15

    2024 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.