Towards Resource-Efficient Compound AI Systems

Adam Belay; Esha Choukse; Gohar Irfan Chaudhry; \'I\~nigo Goiri; Ricardo Bianchini; Rodrigo Fonseca

REVIEW 5 major objections 5 minor 44 references

Towards Resource-Efficient Compound AI Systems

T0 review · 5 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Murakkab claims declarative AI workflows can run 3.4x faster and use 4.5x less energy with unchanged output.

desk verdict Solid vision paper with an honest prototype, but the headline numbers compare Murakkab against a deliberately sequential baseline, so the claimed speedup mostly measures ordinary parallelism. read the letter →

arxiv 2501.16634 v3 pith:JHKIAL7R submitted 2025-01-28 cs.DC cs.AI

Gohar Irfan Chaudhry , Esha Choukse , Íñigo Goiri , Rodrigo Fonseca , Adam Belay , Ricardo Bianchini This is my paper · ORCID

classification cs.DCcs.AI

keywords compoundAIsystemsdeclarativeworkflowsadaptiveruntimeresourceefficiencyworkfloworchestrationclustermanagementenergyfungible

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that compound AI systems waste resources because application logic is fused with fixed model, tool, and hardware choices, and because workflow orchestration and cluster management do not talk to each other. It proposes Murakkab, a runtime that accepts a high-level declarative job description, generates a task graph, maps tasks to interchangeable agents, and picks hardware and execution strategies from measured efficiency and quality profiles. In a video-understanding workflow, this design completed the job in 77 to 83 seconds versus 283 seconds for the imperative baseline, and cut speech-to-text energy from 155 Wh to 34 Wh by running on CPUs instead of GPUs, with identical output. The authors frame this as a first validation of a broader vision in which AI workflows become fungible and resource-efficient without sacrificing result quality.

What carries the argument

The load-bearing mechanism is the pairing of a declarative workflow specification with an adaptive runtime scheduling loop. The orchestrator LLM (NVLM) decomposes the job into a task graph using a ReAct-style approach, maps tasks to agents through a library of model and tool schemas, and the runtime then selects among fungible implementations and hardware configurations using execution profiles that record cost, power, latency, and quality tradeoffs. A separate cluster manager exposes real-time resource availability, and the two layers exchange workflow DAGs and utilization statistics to rebalance resources across agents, enabling parallelism and harvestable-resource decisions that a static workflow cannot make.

What would settle it

Run Murakkab on an unseen multipart job given only its natural-language description, with no subtask hints, and compare the LLM-generated task graph and final output against a hand-authored reference graph and output; any task graph that omits, reorders, or mislabels a required step, or any output that differs in quality, would falsify the claim that declarative abstraction preserves result quality.

Watch

Extended reading notes

Core claim

The paper's central claim is that a Compound AI System can be written declaratively — a natural-language job description, optional subtask hints, and a constraint such as minimum cost — and an adaptive runtime can then lower that description into a directed acyclic graph, map each task to an agent from a library of models and tools, and choose execution configurations such as CPU versus GPU, parallel subtask execution, and extra execution paths. By doing so, Murakkab achieves a roughly 3.4x speedup in workflow completion time and a roughly 4.5x improvement in energy efficiency over a fixed, imperative workflow implementation of the same video-understanding task, while the authors state that the execution output and accuracy are the same in all comparisons. The result is meant to show that efficiency and quality are not inherently in conflict once application logic is decoupled from low-level resource and implementation details.

Load-bearing premise

The load-bearing premise is that an LLM can decompose a natural-language job description into a semantically correct task graph and map each task to a suitable agent; if that decomposition is wrong, the workflow's output changes and the paper's equal-quality claim collapses. The validation uses a simple workflow with subtasks spelled out explicitly, so this hardest part of the design is not yet stressed.

Editorial extensions

If this is right

Developers stop pinning models, tools, and hardware: Murakkab chooses them at runtime from an agent library and per-configuration profiles.
Workflows written declaratively survive model and tool updates without rewriting or reprompting.
Cluster managers gain visibility into the workflow DAG and can move resources between agents based on upcoming demand rather than current utilization.
The same output can be delivered on a continuum of cost, latency, and energy tradeoffs, selected by a constraint such as MIN_COST.
One job can be split into parallel subtasks (for example, per-scene speech-to-text) whenever idle resources exist, which is how the 283-second baseline became a 77-to-83-second run.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: the evidence is one workflow with explicit subtask hints; if automatic decomposition of open-ended job descriptions proves unreliable, the equal-quality claim would need to be scoped to workflows whose task structure is known in advance.
Editorial inference: if this architecture scales, AI workflows could evolve like SQL — a declarative job survives model and hardware churn, and the optimizer absorbs the churn, making the workflow definition itself a durable asset rather than a code artifact to be rewritten.
Editorial inference: the runtime levers suggest a low-cost benchmarking methodology for compound AI systems — report workflows as constraint-optimization problems such as MIN_COST or MIN_LATENCY, and compare the runtime's chosen configuration against the full frontier of model and hardware assignments.
Editorial inference: the CPU-versus-GPU speech-to-text swap is a testable pattern; applying the same substitution to other transcription models and languages could generalize the energy savings, but latency and accuracy shifts would need to be measured per model.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: this is a useful vision paper for resource-efficient compound AI systems, and the authors are upfront that it is preliminary. The new bits are worth naming: a declarative workflow model that hides model and hardware choices, a runtime that does LLM-based decomposition and maps tasks to a library of agents, and explicit integration between workflow orchestration and cluster manager. That last piece, plus treating hardware and model choice as fungible, is what distinguishes it from earlier declarative AI workflow systems like [3,24]. The AIWaaS framing in Section 5 is a nice discussion contribution.

The paper also deserves credit for being honest about overheads and open problems. Table 1 is a clean summary of the levers. The prototyping is real: they ran a video-understanding workflow based on OmAgent, varied Whisper placement, parallelized scenes, and reported energy and time.

Now the soft spots. The stress-test note is on target. The baseline is described as a fixed execution without any intra-task parallelism or opportunity to utilize idle resources, with each scene and frame processed sequentially. That means the 3.4x speedup mostly measures the difference between sequential processing and generic DAG parallelism, not the value of Murakkab's declarative and adaptive machinery. Any competent workflow engine that parallelizes frame summarization and STT across scenes would get much of the same gain. To attribute the gain to Murakkab, they need a strong conventional baseline: the same workflow as an imperative DAG with parallelism enabled. Without that control, the headline claim is unverified. The energy comparison has a related problem: only GPU energy is measured, which is fine if justified, but combined with the baseline issue it overstates the system's efficiency story.

The second load-bearing assumption is LLM-based job decomposition. If the orchestrator LLM produces a semantically wrong DAG, output quality changes. The evaluation workflow is simple, and the authors supply the subtasks in Listing 2, so automatic decomposition from natural language alone is not exercised. The claim that output and accuracy are the same is asserted, not quantified. Also, single run, no error bars, no public artifacts. These are fixable evaluation gaps, not fundamental flaws.

Overall, the central architectural argument—decouple application logic from execution details and let a runtime exploit that flexibility—holds up. The paper's current evidence does not prove the architecture delivers the numbers. It is a solid position paper and a promising research direction.

I would send it to peer review as a vision and systems paper, with the clear expectation that the authors strengthen the evaluation before acceptance: add a realistic parallel baseline, repeat runs, quantify output similarity, and release code and data. If that happens, it could be a useful cite. For now, I would treat the speedup and energy numbers as suggestive, not established.

Referee Report

5 major / 5 minor

Summary. The paper argues that current compound AI systems are inefficient due to tight coupling of application logic with execution configuration, a disconnect between orchestration and cluster management, and a perceived tradeoff between efficiency and quality. It proposes Murakkab, a prototype combining a declarative workflow programming model with an adaptive runtime that uses LLM-based job decomposition, task-to-agent mapping, model/tool selection, resource allocation, and workflow-aware cluster management. The evaluation on a video-understanding workflow reports ~3.4x speedup and ~4.5x higher energy efficiency compared with a sequential OmAgent-derived baseline, with the same output and accuracy asserted. The paper also discusses AI Workflows-as-a-Service (AIWaaS), quality control, proprietary models, and multi-cloud concerns.

Significance. The problem is timely and the architecture is plausible. The paper identifies real inefficiencies in compound AI deployments and proposes a concrete design, with an honest discussion of overheads and open problems. Strengths include the explicit treatment of resource fungibility, the connection to prior cluster-management work, and a preliminary end-to-end prototype. However, the empirical evidence is currently too thin: the evaluation covers one workflow, the baseline is deliberately sequential, no strong parallel baseline is included, output equality is asserted without measurement, and energy is measured only on the GPU. If the evaluation is strengthened, the system could make a useful contribution to the systems community; at present the headline efficiency claims are not fully supported.

major comments (5)

[§4 (Baseline and Murakkab paragraphs)] The reported ~3.4x speedup does not distinguish Murakkab's adaptive runtime from ordinary parallelism. The baseline is described as 'a fixed execution without any intra-task parallelism or opportunity to utilize idle resources,' and the three optimizations attributed to Murakkab—parallel STT across scenes, parallel frame summarization on idle GPUs, and running Whisper on CPUs—are generic dataflow-parallelization and resource-allocation decisions available to any conventional workflow engine. A control that runs the same DAG with a standard parallel orchestrator (and with the same agent choices) is needed to attribute the speedup to the declarative/adaptive design. The current comparison only shows that a sequential pipeline is slower than a parallelized one.
[§4 (first paragraph)] The statement that 'The execution output and accuracy are the same in all comparisons' is not supported by any reported comparison. The paper should define and report a quantitative measure of output equivalence (for example, exact or near-exact transcript match for STT, object detection precision/recall for CLIP, and final answer agreement for the summarization/LLM stages) and show these values for each configuration. Without this, the quality-preservation claim that underpins the fungibility argument is unverified.
[Table 2] The energy-efficiency claim is based on an incomplete measurement. The paper measures only GPU energy, but the headline 4.5x improvement comes from running STT on 64 CPU cores instead of a GPU; excluding CPU energy from that configuration will trivially favor the CPU configuration. The authors should report full node or system energy, or at least include CPU power in the measurement. In addition, Listing 2 specifies MIN_COST, while Table 2 reports energy, not monetary cost; the paper should clarify whether the constraint is cost or energy and report the corresponding metric.
[§3.2 and Listing 2] The evaluation does not exercise the automatic job-decomposition mechanism. Listing 2 supplies the sub-tasks explicitly, and the reported optimizations follow from that explicit task structure. The LLM-based decomposition from a natural-language job description into a DAG, which is a central component of Section 3.2, is therefore untested. The authors should either evaluate a run that starts from only the 'desc' and inputs, or explicitly scope the claims to user-provided task hints.
[Figure 3 and Table 2] The reported times and energies appear to come from single runs; no repetition, error bars, or variance is reported. Given that the speedup and energy claims are central to the paper, at least a few repeated runs with the observed range (or a clear statement that these are single preliminary traces) should be provided.

minor comments (5)

[§2 and §3.3] There are several typos and misspellings; examples include 'throuhgput' (Section 2), 'underprovisionig' (Section 2), 'Task understansing' (Section 3.3), and 'Worfklow' in the Section 3.2 heading. The paper needs a careful copyedit.
[References] Reference [8] is mislabeled: the title 'What is provisioned throughput?' does not match the Azure Spot VMs URL; please point to the correct documentation.
[§3.3] The statement that DAG-creation LLM queries take 'less than 1% of the execution time' is not backed by a measurement or a derivation; please report the measurement or label it as an estimate.
[Table 1] The 'CPU vs GPU' row reports 'Lower/No Change' for quality, which is ambiguous. Please clarify the conditions under which moving a model to CPU changes quality (for example, numerical precision or latency effects) and whether this is a single selection.
[§4] The baseline completion time is given as 283 s in the text and 285 s in Table 2; please reconcile.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the central claims are empirical measurements against an explicit baseline, not derivations from fitted inputs or self-citation chains.

full rationale

The paper does not contain fitted parameters that are renamed as predictions, and it does not derive its speedup or energy-efficiency numbers from its own definitions. The reported ~3.4x speedup and ~4.5x energy improvement are measured execution traces of a concrete prototype (Figures 3 and Table 2). The baseline is described explicitly as a fixed sequential execution without intra-task parallelism, so the comparison is a controlled experiment rather than a quantity that equals its input by construction. The paper also states that the execution output and accuracy are the same in all comparisons, which may be under-evidenced but is not a circular step. The LLM-based job decomposition is not load-bearing in the evaluation because Listing 2 provides explicit sub-tasks; and while the paper cites prior work co-authored by some of the same researchers (Quicksand [36], Harvest VMs [2]), these citations provide background concepts and are not invoked as a load-bearing uniqueness theorem or as a substitute for the measured results. Thus there is no reduction of the central claim to its inputs, and the derivation chain is self-contained.

Assumptions & free parameters 0 free parameters · 4 assumptions · 2 invented entities

The central claims rest mostly on the assumed reliability of LLM-based decomposition and on execution profiles. These are domain assumptions, not formally validated in the paper. No free parameters are fitted to data; the workflow configurations compared (CPU vs GPU for STT) are system levers, not fitted constants. The prototype itself and the AIWaaS model are new postulated artifacts whose only evidence is the preliminary evaluation in this paper.

assumptions (4)

domain assumption The orchestrator LLM reliably decomposes job descriptions into correct task graphs (DAGs) and maps tasks to agents.
Invoked in Section 3.2 'Job Decomposition'; if decomposition is wrong, workflow output differs from intended, violating the claim that quality is unchanged.
domain assumption Execution profiles of each model/tool on each hardware pair accurately represent real runtime, cost, and power.
Section 3.2 'Model/Tool Selection'; all runtime decisions are based on these profiles, which are measured offline and assumed stable.
domain assumption Alternative models and tools in the library are functionally substitutable at target quality.
Section 3.1 and 3.2 describe dynamic selection of models and tools; if substitutes change output quality, the quality-preservation promise fails.
domain assumption Workflow orchestrator and cluster manager can share state and act jointly without prohibitive overhead or conflicting objectives.
Section 3.2 'Workflow-Aware Cluster Management' presupposes the integration is feasible and beneficial; the paper provides no end-to-end implementation proof.

invented entities (2)

Murakkab adaptive runtime
purpose: Selects models, tools, hardware, and parallelism dynamically to optimize cost, energy, and latency.
The system is under construction; its benefits are demonstrated only in this paper's preliminary evaluation on a single workflow, with no external or independent validation yet.
AI Workflows-as-a-Service (AIWaaS)
purpose: Proposed service model where developers submit high-level job descriptions and the platform manages everything below.
Vision only; no implementation or evaluation provided in this paper.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Towards Resource-Efficient Compound AI Systems." pith.science (2026). https://pith.science/paper/JHKIAL7R

@misc{pith2026250116634,
  author       = {Pith},
  title        = {Pith review of: Towards Resource-Efficient Compound AI Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JHKIAL7R}},
  note         = {Machine review of arXiv:2501.16634}
}

abstract

Compound AI Systems, integrating multiple interacting components like models, retrievers, and external tools, have emerged as essential for addressing complex AI tasks. However, current implementations suffer from inefficient resource utilization due to tight coupling between application logic and execution details, a disconnect between orchestration and resource management layers, and the perceived exclusiveness between efficiency and quality. We propose a vision for resource-efficient Compound AI Systems through a declarative workflow programming model and an adaptive runtime system for dynamic scheduling and resource-aware decision-making. Decoupling application logic from low-level details exposes levers for the runtime to flexibly configure the execution environment and resources, without compromising on quality. Enabling collaboration between the workflow orchestration and cluster manager enables higher efficiency through better scheduling and resource management. We are building a prototype system, called Murakkab, to realize this vision. Our preliminary evaluation demonstrates speedups up to $\sim 3.4\times$ in workflow completion times while delivering $\sim 4.5\times$ higher energy efficiency, showing promise in optimizing resources and advancing AI system design.

Figures

Figures reproduced from arXiv: 2501.16634 by the authors.

**Figure 1.** Today programmers use frameworks to call agents from different providers hosted on multiple cloud platforms. The rigid coupling between all layers of the system results in inefficiencies. OpenCV (Tool) CLIP (ML model) Whisper (ML model) Llama (LLM) Declarative Programming Model Model/Tool Library Cloud Platform(s) A100 GPUH100 GPU A100 GPUA100 GPUA100 GPU Resource Stats Calculator (Tool) SigLIP (ML model) Web Search… view at source ↗

**Figure 2.** We envision fungible workflows with high-level descriptions, managed jointly by the Workflow Orchestrator and Cluster Manager. This allows higher resource multiplexing between independent workflows to improve efficiency. and provide memory to stateless models etc. Examples include LangChain [23], LlamaIndex [25] and DSPy [38]. Agent Providers offer specific models, tools, or vector databases etc. typically throug… view at source ↗

**Figure 3.** Execution traces of the Video Understanding work [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

44 extracted references · 23 canonical work pages

[1]

Amazon Web Services. 2025. Amazon Web Services (AWS). https: //aws.amazon.com/

work page 2025
[2]

Pradeep Ambati, Íñigo Goiri, Felipe Frujeri, Alper Gun, Ke Wang, Brian Dolan, Brian Corell, Sekhar Pasupuleti, Thomas Moscibroda, Sameh Elnikety, et al. 2020. Providing SLOs for Resource-Harvesting VMs in cloud platforms. In OSDI

work page 2020
[3]

Shah, Benjamin Sowell, Dan Tecuci, Vinayak Thapliyal, and Matt Welsh

Eric Anderson, Jonathan Fritz, Austin Lee, Bohou Li, Mark Lindblad, Henry Lindeman, Alex Meyer, Parth Parmar, Tanvi Ranade, Mehul A. Shah, Benjamin Sowell, Dan Tecuci, Vinayak Thapliyal, and Matt Welsh. 2024. The Design of an LLM-powered Unstructured Analytics System. arXiv:2409.00847 [cs.DB] https://arxiv.org/abs/2409.00847

arXiv 2024
[4]

The Kubernetes Authors. 2024. Kubernetes Documentation. https: //kubernetes.io/

work page 2024
[5]

Microsoft Azure. [n.d.]. GPU Accelerated Virtual Machines: NDm A100 v4-Series. https://learn.microsoft.com/en-us/azure/virtual-machines/ sizes/gpu-accelerated/ndma100v4-series?tabs=sizebasic

work page
[6]

Microsoft Azure. 2024. Azure Machine Learning. https://azure. microsoft.com/en-us/products/machine-learning

work page 2024
[7]

Microsoft Azure. 2024. What is provisioned throughput? https://learn.microsoft.com/en-us/azure/ai-services/openai/ concepts/provisioned-throughput

work page 2024
[8]

Microsoft Azure. 2024. What is provisioned throughput? https: //azure.microsoft.com/en-us/products/virtual-machines/spot/

work page 2024

Show all 44 references

[9]

G. Bradski. 2000. The OpenCV Library. Dr. Dobb’s Journal of Software Tools (2000)

2000
[10]

Surajit Chaudhuri. 1998. An overview of query optimization in relational systems. In Proceedings of the Seventeenth ACM SIGACT- SIGMOD-SIGART Symposium on Principles of Database Systems (Seattle, Washington, USA) (PODS ’98). Association for Computing Machinery, New York, NY, U...

1998
[11]

Lingjiao Chen, Matei Zaharia, and James Zou. 2023. FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performance. arXiv:2305.05176 [cs.LG] https://arxiv.org/abs/2305. 05176

2023 arXiv
[12]

NVIDIA Corporation. 2022. NVIDIA H100 Tensor Core GPU Datasheet. https://resources.nvidia.com/en-us-tensor-core/nvidia- tensor-core-gpu-datasheet

2022
[13]

Wenliang Dai, Nayeon Lee, Boxin Wang, Zhuolin Yang, Zihan Liu, Jon Barker, Tuomas Rintamaki, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. 2024. NVLM: Open Frontier-Class Multimodal LLMs. arXiv:2409.11402 https://arxiv.org/abs/2409.11402

2024 arXiv
[14]

Databricks. 2025. Databricks Large Language Model Serving. https: //docs.databricks.com/en/large-language-models/index.html

2025
[15]

Christina Delimitrou and Christos Kozyrakis. 2013. Paragon: QoS- aware scheduling for heterogeneous datacenters. In Proceedings of the Eighteenth International Conference on Architectural Support for Programming Languages and Operating Systems (Houston, Texas, USA) (ASPLOS ’13...

2013
[16]

Christina Delimitrou and Christos Kozyrakis. 2014. Quasar: resource- efficient and QoS-aware cluster management. In Proceedings of the 19th International Conference on Architectural Support for Programming Languages and Operating Systems (Salt Lake City, Utah, USA) (ASP- LOS ’...

2014
[17]

Google Cloud. 2025. Google Cloud Platform. https://cloud.google. com/

2025
[18]

Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhi- nav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, et al . 2024. The Llama 3 Herd of Models. arXiv:2407.21783 [cs.CL] https://arxiv.org/abs/2407.21783

2024 arXiv
[19]

Goucher, Adam Perelman, Aditya Ramesh, et al

Aaron Hurst, Adam Lerer, Adam P. Goucher, Adam Perelman, Aditya Ramesh, et al. 2024. GPT-4o System Card. arXiv:2410.21276 [cs.CL] https://arxiv.org/abs/2410.21276

2024 arXiv
[20]

Matthias Jarke and Jurgen Koch. 1984. Query Optimization in Database Systems. ACM Comput. Surv. 16, 2 (June 1984), 111–152. https://doi. org/10.1145/356924.356928

1984
[21]

Eric Jonas, Johann Schleier-Smith, Vikram Sreekanti, Chia-Che Tsai, Anurag Khandelwal, Qifan Pu, Vaishaal Shankar, Joao Carreira, Karl Krauth, Neeraja Yadwadkar, et al . 2019. Cloud programming sim- plified: A berkeley view on serverless computing. arXiv preprint arXiv:1902.03...

2019 arXiv
[22]

Norman P. Jouppi, George Kurian, Sheng Li, Peter Ma, Rahul Nagara- jan, Lifeng Nai, Nishant Patil, Suvinay Subramanian, Andy Swing, Brian Towles, Cliff Young, Xiang Zhou, Zongwei Zhou, and David Patterson. 2023. TPU v4: An Optically Reconfigurable Supercom- puter for Machine L...

2023 arXiv
[23]

LangChain. 2024. LangChain. https://github.com/langchain-ai/ langchain

2024
[24]

Chunwei Liu, Matthew Russo, Michael Cafarella, Lei Cao, Peter Baille Chen, Zui Chen, Michael Franklin, Tim Kraska, Samuel Madden, and Gerardo Vitagliano. 2024. A Declarative System for Optimizing AI Workloads. arXiv:2405.14696 [cs.CL]

2024 arXiv
[25]

Jerry Liu. 2022. LlamaIndex. https://github.com/jerryjliu/llama_index

2022
[26]

Samuel Madden, Michael Cafarella, Michael Franklin, and Tim Kraska
[27]

Microsoft. 2025. Microsoft Azure. https://azure.microsoft.com/

2025
[28]

NVIDIA. 2024. NVIDIA A100 Tensor Core GPU Datasheet. https://www.nvidia.com/content/dam/en-zz/Solutions/Data-Center/ a100/pdf/nvidia-a100-datasheet-us-nvidia-1758950-r4-web.pdf

2024
[29]

OpenAI. 2023. Function Calling Guide. https://platform.openai.com/ docs/guides/function-calling

2023
[30]

OpenAI. 2023. OpenAI API Reference. https://platform.openai.com/ docs/api-reference/introduction

2023
[31]

OpenAI. 2025. OpenAI Large Language Models and API. https:// platform.openai.com/docs/

2025
[32]

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Saurabh Sastry, Amanda Askell, Pam Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. https: //openai....

2021
[33]

Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2022. Robust Speech Recognition via Large-Scale Weak Supervision. https://openai.com/research/whisper. OpenAI

2022
[34]

Dima Rekesh, Nithin Rao Koluguri, Samuel Kriman, Somshubra Ma- jumdar, Vahid Noroozi, He Huang, Oleksii Hrinchuk, Krishna Puvvada, Ankur Kumar, Jagadeesh Balam, and Boris Ginsburg. 2023. Fast Con- former with Linearly Scalable Attention for Efficient Speech Recogni- tion. arXi...

2023 arXiv
[35]

Francisco Romero and Christina Delimitrou. 2018. Mage: online and interference-aware scheduling for multi-scale heterogeneous systems.6 In Proceedings of the 27th International Conference on Parallel Archi- tectures and Compilation Techniques (Limassol, Cyprus) (PACT ’18). Ass...

2018
[36]

Aguilera, Adam Belay, Seo Jin Park, and Malte Schwarzkopf

Zhenyuan Ruan, Shihang Li, Kaiyan Fan, Marcos K. Aguilera, Adam Belay, Seo Jin Park, and Malte Schwarzkopf. 2023. Unleashing True Utility Computing with Quicksand. InProceedings of the 19th Workshop on Hot Topics in Operating Systems (Providence, RI, USA) (HOTOS ’23). Associat...

2023
[37]

Amazon Web Services. 2024. Amazon SageMaker: Build, Train, and Deploy Machine Learning Models at Scale. https://aws.amazon.com/ sagemaker/

2024
[38]

Stanford NLP Group. 2023. DSPy: The Framework for Pro- gramming—Not Prompting—Language Models. https://github.com/ stanfordnlp/dspy

2023
[39]

Ion Stoica and Scott Shenker. 2021. From Cloud Computing to Sky Computing. In HotOS. https://doi.org/10.1145/3458336.3465301

2021
[40]

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2023. Chain- of-Thought Prompting Elicits Reasoning in Large Language Models. arXiv:2201.11903 [cs.CL] https://arxiv.org/abs/2201.11903

2023 arXiv
[41]

Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing Reasoning and Acting in Language Models. arXiv:2210.03629 [cs.CL] https://arxiv. org/abs/2210.03629

2023 arXiv
[42]

Matei Zaharia, Omar Khattab, Lingjiao Chen, Jared Quincy Davis, Heather Miller, Chris Potts, James Zou, Michael Carbin, Jonathan Frankle, Naveen Rao, and Ali Ghodsi. 2024. The Shift from Models to Compound AI Systems. https://bair.berkeley.edu/blog/2024/02/18/ compound-ai-systems/

2024
[43]

Lu Zhang, Tiancheng Zhao, Heting Ying, Yibo Ma, and Kyu- song Lee. 2024. OmAgent: A Multi-modal Agent Framework for Complex Video Understanding with Task Divide-and-Conquer. arXiv:2406.16620 [cs.CL] https://arxiv.org/abs/2406.16620 7

2024 arXiv
[2024]

Databases Unbound: Querying All of the World’s Bytes with AI. Proc. VLDB Endow. 17, 12 (Nov. 2024), 4546–4554. https://doi.org/10. 14778/3685800.3685916

2024

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Amazon Web Services. 2025. Amazon Web Services (AWS). https: //aws.amazon.com/

work page 2025

[2] [2]

Pradeep Ambati, Íñigo Goiri, Felipe Frujeri, Alper Gun, Ke Wang, Brian Dolan, Brian Corell, Sekhar Pasupuleti, Thomas Moscibroda, Sameh Elnikety, et al. 2020. Providing SLOs for Resource-Harvesting VMs in cloud platforms. In OSDI

work page 2020

[3] [3]

Shah, Benjamin Sowell, Dan Tecuci, Vinayak Thapliyal, and Matt Welsh

Eric Anderson, Jonathan Fritz, Austin Lee, Bohou Li, Mark Lindblad, Henry Lindeman, Alex Meyer, Parth Parmar, Tanvi Ranade, Mehul A. Shah, Benjamin Sowell, Dan Tecuci, Vinayak Thapliyal, and Matt Welsh. 2024. The Design of an LLM-powered Unstructured Analytics System. arXiv:2409.00847 [cs.DB] https://arxiv.org/abs/2409.00847

arXiv 2024

[4] [4]

The Kubernetes Authors. 2024. Kubernetes Documentation. https: //kubernetes.io/

work page 2024

[5] [5]

Microsoft Azure. [n.d.]. GPU Accelerated Virtual Machines: NDm A100 v4-Series. https://learn.microsoft.com/en-us/azure/virtual-machines/ sizes/gpu-accelerated/ndma100v4-series?tabs=sizebasic

work page

[6] [6]

Microsoft Azure. 2024. Azure Machine Learning. https://azure. microsoft.com/en-us/products/machine-learning

work page 2024

[7] [7]

Microsoft Azure. 2024. What is provisioned throughput? https://learn.microsoft.com/en-us/azure/ai-services/openai/ concepts/provisioned-throughput

work page 2024

[8] [8]

Microsoft Azure. 2024. What is provisioned throughput? https: //azure.microsoft.com/en-us/products/virtual-machines/spot/

work page 2024