Pith. sign in

REVIEW 3 major objections 4 minor 10 cited by

This paper claims that serving agentic workflows as declarative DAGs, instead of opaque hard-coded pipelines, lets a profile-guided optimizer cut GPU usage by up to 2.8x, energy by 3.7x, and cost by 4.3x while preserving quality and latency

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Murakkab uses declarative workflow specs and a profile-guided MILP optimizer to reduce GPU, energy, and cost for agentic workflow serving while meeting percentile-defined SLOs.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection A promising integrated system for agentic workflow serving whose efficiency gains are real but overstated—the SLO guarantee is definitional and the baseline is hand-crafted. the 3 major comments →

arxiv 2508.18298 v2 pith:IBXH3FIA submitted 2025-08-22 cs.MA cs.AIcs.SE

Murakkab: Resource-Efficient Agentic Workflow Orchestration in Cloud Platforms

Gohar Irfan Chaudhry , Esha Choukse , Haoran Qiu , \'I\~nigo Goiri , Rodrigo Fonseca , Adam Belay , Ricardo Bianchini This is my paper
classification cs.MA cs.AIcs.SE
keywords agentic workflowsLLM servingdeclarative orchestrationprofile-guided optimizationmixed-integer linear programmingSLO-aware schedulingmulti-tenant resource managementenergy-efficient AI
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that agentic workflows are served inefficiently because orchestration frameworks expose them as opaque sequences of model and tool calls, hard-coding model and hardware choices and fragmenting configuration decisions across developers, providers, and platforms. It proposes Murakkab, a serving system with a declarative workflow abstraction that separates what a workflow does from how it is executed, letting a profile-guided optimizer and an adaptive runtime jointly choose workflow knobs, models, hardware, and parallelism. The central claim is that this end-to-end visibility unlocks cross-layer optimizations that black-box serving cannot achieve, and the evaluation reports up to 2.8x lower GPU usage, 3.7x lower energy, and 4.3x lower cost while sustaining quality and latency SLOs on production-shaped traces. If correct, agent developers would no longer need to hand-tune models and accelerators, and cloud platforms could reclaim significant resources without degrading user-visible outcomes.

Core claim

The core discovery is that the internal structure of an agentic workflow is a reusable optimization surface. By exposing workflows as typed DAGs of logical tasks, Murakkab can decouple the developer's intent from execution details and then solve a joint optimization problem: pick workflow-level knobs such as frame counts or debate rounds, map each task to a model or tool, choose GPU type and tensor parallelism, allocate instance counts, and multiplex load across workflows and SLO tiers. Offline profiles make this search tractable, with one layer for workflow quality and token load and another for model latency, energy, and cost under different hardware configurations. Every hour the optimize

What carries the argument

The central mechanism is the pairing of offline profile layers with a mixed-integer linear program (MILP), a mathematical optimization formulation that allocates resources under constraints. Workflow profiles record, per configuration, the response quality and per-executor token load; model profiles record time-to-first-token, time-per-output-token, energy, and cost across GPU types and tensor-parallelism settings. The MILP consumes profiles, arrival patterns, and resource constraints, separates peak provisioning from average utilization, and outputs instance counts and workflow-to-model assignments, while a runtime auto-scaler corrects short-term prediction error. This mechanism makes the e

Load-bearing premise

The evaluation treats a fixed, hand-chosen static GPU allocation as the representative production baseline and defines SLO tiers as percentiles of the profiled configuration set; if real deployments already adapt allocation to load or users require absolute SLO thresholds, the reported savings and the 'maintains SLOs' guarantee may not transfer.

What would settle it

Run the same production-derived 24-hour request mix on an imperative orchestration framework enhanced with dynamic autoscaling and per-request configuration selection, then compare total GPU-hours, energy, and the number of SLO violations. If the adaptive baseline uses no more GPUs and still meets all SLOs, the central claim that workflow-structure visibility is what unlocks the savings would be falsified.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Per-request SLO tiers become first-class: a request asking for a 'good' accuracy or latency tier can be served by a materially cheaper configuration than a 'best' request, and the system enforces that difference.
  • Hourly reoptimization plus an auto-scaler lets the platform track diurnal demand, so GPU counts, model instances, and workflow configurations change with load rather than staying fixed.
  • Multiplexing different workflows onto shared model instances yields additional 17-21% reductions beyond per-workflow optimization in the reported setup.
  • Workflow DAG visibility lets the scheduler place cheap components on CPUs and reserve GPUs for latency-critical ones, meeting a 30-second SLO with fewer GPUs.
  • A roughly 60-minute optimization epoch emerges as a balanced operating point: shorter intervals are dominated by transition overhead, longer intervals by demand-prediction error.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The reported multiples compare against a fixed, hand-picked static allocation; against a baseline that already autoscales or dynamically selects models, the savings would likely shrink because the design's real advantage is joint cross-layer optimization, not adaptation alone.
  • The same profile-guided decoupling could extend to workflows whose components span multiple cloud providers or proprietary APIs, treating external endpoints as just another model profile with cost, latency, and accuracy attributes.
  • Because SLO tiers are defined as percentiles of the profiled configuration set, the meaning of 'best' shifts as new models are added; this makes the system self-improving but also means tiers are relative to the current library, not an absolute user contract.
  • The hidden cost is profile freshness: as models and hardware churn, the profiles must be re-measured, and the paper's claims assume those profiles stay accurate throughout the 24-hour evaluation window.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper presents Murakkab, a serving system for agentic workflows that decouples declarative workflow specifications from execution configuration, uses offline profiles and a MILP optimizer to select workflow/model/hardware knobs per SLO tier, and includes an adaptive runtime and auto-scaler for multi-tenant cloud platforms. The evaluation on video Q/A, code generation, and math Q/A workflows over 24-hour Azure traces reports up to 2.8x GPU reduction, 3.7x energy reduction, and 4.3x cost reduction relative to a static hand-crafted baseline, while claiming no SLO violations.

Significance. If the results held, Murakkab would be a substantial contribution to compound AI serving: exposing workflow DAGs to a profile-guided optimizer is a timely and plausible direction. The MILP formulation in Appendix A.5 is internally consistent, and the sensitivity analysis in Section 4.6 is a useful practical study. Credit is also due for the paper's transparency about the static baseline's limitations. However, the central efficiency and SLO-maintenance claims are not established by the evaluation as written: the SLO tiers are defined over the optimizer's own configuration set, the latency filter ignores DAG-level critical paths, and the only comparison is to a hand-crafted, non-adaptive baseline that itself violates SLOs. The idea deserves further development, but the current evidence does not support the headline numbers.

major comments (3)
  1. [Section 3.4 and Appendix A.5, Eqs. (4)-(5), (8)-(9)] The SLO tiers are defined as the best/95th/80th/50th percentiles of accuracy and latency among the profiled workflow/model/hardware configuration set. The MILP's SLO-filtering constraints then reject any configuration that falls below these self-referential thresholds. Consequently, every feasible solution satisfies the SLO by construction, and the 'no SLO violations' claim in Figure 10b is a design property, not an empirical result. To substantiate SLO maintenance, the evaluation must use absolute user-defined SLO targets or externally specified thresholds that are not derived from the optimizer's own profile distribution.
  2. [Appendix A.5, Eq. (5) and Eq. (9); Section 4.5] The only latency filter in the MILP rejects configuration c on model m when TTFT_m + t_c * TPOT_m > tau. This is a single-model generation-latency check. It does not model the DAG critical path: tool execution (frame extraction, Whisper, OmDet), parallel branches, queueing, or colocation effects. Section 4.5's own example shows end-to-end latency is determined by where tools run and how parallel branches overlap, not by one model's TTFT/TPOT. Thus the optimizer can admit configurations that violate the end-to-end latency SLO, so the claimed guarantee is not actually enforced by the formulation.
  3. [Section 4.1 and Table 1] The efficiency numbers are relative to a hand-crafted static baseline that uses a fixed Gemma-3-27B-on-A100 allocation for 24 hours, does not adapt to load, and does not distinguish SLO tiers. Calling this representative of 'state-of-the-art baselines like LangGraph' conflates an orchestration framework with a deployment policy; real deployments of LangChain/LangGraph can be run with autoscalers, and an autoscaling or SLO-aware baseline would likely close a large part of the reported gap. Since the baseline also violates SLOs (Figure 10b), the reported 2.8x/3.7x/4.3x improvements are not a fair measure of improvement over current practice.
minor comments (4)
  1. [Appendix A.5] Constraints (4)-(5) and (8)-(9) are duplicated verbatim, and the cost budget constraint appears as (6) and again as (10). This makes the formulation harder to read.
  2. [Section 4.3] Typo: 'itsholistic' should read 'its holistic'.
  3. [Section 4.1] The description of the static policy as 'hand-crafted' is clear, but the claim that it 'represents existing systems like LangGraph' is not supported, as LangGraph does not prescribe a static GPU allocation.
  4. [Figure 10b] The legend includes 'SLO' but the plot does not clearly label which bar/line corresponds to the SLO threshold, making it hard to visually verify the claimed violations.

Circularity Check

1 steps flagged

SLO compliance is definitional: Section 3.4 defines SLO tiers as percentiles of the optimizer's own configuration set, and the MILP filters to those percentiles, so the 'no SLO violations' claim is enforced by construction.

specific steps
  1. self definitional [Section 3.4 (Execution, SLOs), enforced by Appendix A.5 constraints (4)-(5) and (8)-(9)]
    "We assign four SLO tiers for quality and end-to-end latency: best, good, fair, and basic. The SLO tiers correspond to the best, 95th, 80th, and 50th percentile values of accuracy and latency available among the set of all workflow, model, and hardware configurations."

    The SLO thresholds tau_{w,s} are defined as percentiles of accuracy and latency computed over the same configuration set C_w that the optimizer searches. The MILP then zeroes out any configuration whose accuracy is below tau (constraints 4/8) or whose latency estimate exceeds tau (constraints 5/9). Consequently, every feasible Murakkab allocation is SLO-compliant by construction: the filter removes configurations outside the chosen quantile, so 'maintaining SLOs' (and the absence of SLO violations in Figures 10b/17b) is a restatement of the definition, not an empirically tested outcome. The reported 2.8x/3.7x/4.3x savings in GPUs/energy/cost are still genuine resource-allocation comparisons against the static baseline, so the paper is only partially circular; but the 'while maintaining SLO

full rationale

The core efficiency results are not circular: the optimizer solves the MILP in Appendix A.5 against measured workflow and model profiles, and the GPU/energy/cost reductions are compared with a fixed static allocation; the Azure trace is external data, albeit partly author-affiliated, and no load-bearing uniqueness theorem or ansatz-smuggling citation appears. However, the companion claim of 'maintaining SLOs' is partly definitional. Section 3.4 defines the four SLO tiers as the best/95th/80th/50th percentiles of accuracy and latency over the same configuration set the optimizer explores, and Appendix A.5's constraints simply discard configurations below those percentiles (Eqs. 4-5 and 8-9). Any feasible allocation therefore satisfies the internally defined SLO by construction; the 'no SLO violations' reported for Murakkab in Figures 10b and 17b is not an independently measured outcome. In addition, the latency filter uses a single-model TTFT + tokens*TPOT formula rather than the DAG critical path, tool execution, or queueing/colocation effects, so even the defined latency feasibility is asserted through the optimizer's proxy rather than verified end-to-end. These issues do not invalidate the resource-efficiency comparisons, but they make the SLO-maintenance claim tautological with respect to the paper's own SLO definition. The hand-crafted static baseline is a comparison-validity concern, not a circularity. Overall: partial circularity, score 6.

Axiom & Free-Parameter Ledger

7 free parameters · 5 axioms · 0 invented entities

The central evaluation rests on measured profiles, trace assumptions, and several hand-chosen parameters. The most consequential are the SLO-tier percentiles, which make SLO compliance definitional, and the unspecified multiplexing factors mu_m, which are required to compute capacity. No new physical or formal entities are introduced.

free parameters (7)
  • Unified buffer factor alpha = 1.15
    Chosen by hand in Appendix A.5 constraints (1)-(2); no sensitivity analysis.
  • EWMA smoothing alpha = 0.5
    Section 4.6, chosen for demand prediction; no tuning analysis.
  • Model-specific multiplexing factor mu_m
    Appendix A.5 constraint (3); values not provided anywhere, yet they directly scale the peak capacity constraint.
  • Optimization epoch = 60 minutes
    Section 3.4 and 4.6; sensitivity analysis shows trade-offs, but the choice is tuned on the same simulation.
  • SLO tier percentiles = best, 95th, 80th, 50th
    Section 3.4; defines the SLO targets relative to the configuration set, making SLO satisfaction a design property.
  • Provisioning delay = 20 minutes
    Section 4.6; assumed from refs [28,33,56], drives the buffer cost analysis.
  • Epsilon in accuracy objective = 0.001
    Appendix A.5 objective (13); small regularization constant, minor but free.
axioms (5)
  • domain assumption Azure chat/coding traces represent agentic workflow arrival patterns.
    Section 4.1 and Appendix A.4 map chat requests to video Q/A and coding requests to code generation; no agentic-workflow trace exists, so the validity of this proxy is assumed.
  • domain assumption Offline profiles accurately predict live accuracy, latency, energy, and cost.
    Section 3.3 profiles are used as 'structured priors' for the MILP; the evaluation assumes these measurements transfer to the simulated 24h run.
  • domain assumption An LLM with tool-calling can map declarative tasks to suitable executors.
    Section 3.2 orchestrator relies on LLM-based executor selection with type-checking; no error rates or robustness data are provided.
  • ad hoc to paper The statistical multiplexing model is accurate.
    Appendix A.5 constraint (3) introduces mu_m without specifying how it is derived; the capacity result depends on it.
  • ad hoc to paper Static allocations with no adaptation represent current practice.
    Section 4.1 defines the Static baseline 'to represent existing systems like LangGraph'; this is an assumption about the state of the art that is not validated.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Murakkab: Resource-Efficient Agentic Workflow Orchestration in Cloud Platforms." pith.science (2026). https://pith.science/paper/IBXH3FIA

@misc{pith2026250818298,
  author       = {Pith},
  title        = {Pith review of: Murakkab: Resource-Efficient Agentic Workflow Orchestration in Cloud Platforms},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IBXH3FIA}},
  note         = {Machine review of arXiv:2508.18298}
}
Share X Bluesky LinkedIn Reddit HN
abstract

Agentic workflows commonly coordinate multiple models and tools with complex control logic. They are quickly becoming the dominant paradigm for AI applications. However, serving them remains inefficient with today's frameworks. The key problem is that they expose workflows as opaque sequences of model and tool calls that tightly couple agent logic with model and hardware choices. Often, these workflow components are fragmented across different entities, preventing systems from reasoning about trade-offs across accuracy, latency, energy, and cost. This leads to resource waste and degraded service-level objectives (SLOs). We present Murakkab, a resource-efficient serving system for agentic workflows. Murakkab introduces a declarative abstraction that decouples workflow specification from execution configuration. A profile-guided optimizer and adaptive runtime jointly manage the full stack: orchestrating workflow components, mapping them to models and hardware, and dynamically reconfiguring execution to satisfy user-defined SLOs. By exposing the internal structure of agentic workflows, Murakkab enables cross-layer optimization that existing frameworks and cloud schedulers cannot achieve. Our evaluation on diverse workflows shows that Murakkab reduces GPU usage by up to 2.8$\times$, energy consumption by 3.7$\times$, and cost by 4.3$\times$ while maintaining SLOs.

Figures

Figures reproduced from arXiv: 2508.18298 by Adam Belay, Esha Choukse, Gohar Irfan Chaudhry, Haoran Qiu, \'I\~nigo Goiri, Ricardo Bianchini, Rodrigo Fonseca.

Figure 1
Figure 1. Figure 1: Today workflow developers use frameworks to call agents from different providers hosted on multiple cloud platforms. This fragmentation results in inefficiencies. layer pursues different goals (latency, quality, cost, or utiliza￾tion), but coordination is minimal, leading to inefficiencies: 1. Tight coupling: Hard-coded parameters (e.g., models) and hardware choices hinder automated optimization. 2. Disjoi… view at source ↗
Figure 2
Figure 2. Figure 2: Two agentic workflows with different characteristics and components. We evaluate Murakkab on representative workflows (video question answering, code generation and mathematical prob￾lem solving) using production-scale traces. Murakkab achieves up to 2.8× lower GPU usage, 3.7× less energy, and 4.3× lower cost than state-of-the-art baselines like LangGraph while pre￾serving quality and latency SLOs. Summary… view at source ↗
Figure 3
Figure 3. Figure 3: Workflow accuracy under different configurations and the token generation load on the respective models. 0 500 1000 1500 2000 2500 0.00 0.05 0.10 TPOT P90 (s) DeepSeek-Qwen-32B 0 500 1000 1500 2000 2500 0 1 2 TTFT P90 (s) DeepSeek-Qwen-32B 0 500 1000 1500 2000 2500 0 25 50 TPS per Wh DeepSeek-Qwen-32B 500 1000 1500 2000 2500 0.00 0.05 0.10 TPOT P90 (s) Gemma-3-27B 500 1000 1500 2000 2500 0 1 2 TTFT P90 (s)… view at source ↗
Figure 4
Figure 4. Figure 4: Model performance under different hardware and parallelism configurations. Some configurations, such as DeepSeek-Qwen-32B [24] (a reasoning model), achieve the highest accuracy but at a much higher token generation cost (Figure 3d). At the median, it generates ≈20,000 tokens versus ≈2,500 for Gemma-3-27B under the same workflow configuration. Yet token counts alone do not capture the full end-to-end workfl… view at source ↗
Figure 5
Figure 5. Figure 5: Large space of workflow configurations along a subset of knobs and metrics. Orchestrator Logical Workflow Workflow Spec. workflow(): ... Executor Library Workflow Dev. (a) Development phase. Workflow Registry WF 1 Logical Optimizer Workflows (SLOs + Demand) Model-to-H/W Models + H/W Allocation Executable Workflows WF 2 Profiles (Workflow + Model) (b) Optimization and deployment phase. Workflow Models + H/W… view at source ↗
Figure 6
Figure 6. Figure 6: Murakkab manages end-to-end workflow life-cycle: from development to optimized deployment and execution. 1 # ===== Sub-tasks in the workflow ===== 2 scene_detect = "Given a list of videos, identify scenes in each." 3 frame_extract = "Given a list of scenes, extract frames." 4 stt = "Given a list of scenes, convert audio to text." 5 q_a = "Answer the query given some context." 6 # ===== Workflow description… view at source ↗
Figure 7
Figure 7. Figure 7: Summary of Murakkab’s optimization process. It combines workflow and model profiles to estimate la￾tency, cost, and energy for candidate configurations. Work￾flow profiles capture task accuracy and executor-level load; model profiles quantify latency, energy, and cost under vary￾ing load and system setups. For requests with SLOs on ac￾curacy, latency, or cost, the optimizer selects configurations that meet… view at source ↗
Figure 8
Figure 8. Figure 8: Video Q/A workflow configured for different SLOs and optimization objectives. 5 10 A100 H100 0 1 200 300 0 25 500 750 Best 91.4% Good 88.9% Fair 87.1% Basic 75.5% 0 50 Min Energy Min Cost # of GPUs (x1000) Energy (MWh) Cost (x1000 $) (a) Accuracy SLOs. 5 8 A100 H100 0 1 150 200 0 5 500 1000 Best 11.3s Good 25.5s Fair 35.3s Basic 78.2s 0 50 Min Energy Min Cost # of GPUs (x1000) Energy (MWh) Cost (x1000 $) (… view at source ↗
Figure 9
Figure 9. Figure 9: Code generation workflow configured for different SLOs and optimization objectives. deviations in workload or resource usage, triggers early re￾optimization to adapt quickly. 4 Evaluation 4.1 Experimental Setup Hardware. We run our experiments on A100 and H100 VMs from Microsoft Azure. Each A100 VM has 8×NVIDIA A100 (80GB) GPUs and an AMD EPYC 7V12 64-Core pro￾cessor, while each H100 VM has 8×NVIDIA H100 (… view at source ↗
Figure 10
Figure 10. Figure 10: Comparing three policies: (1) a hand-crafted static configuration, (2) Murakkab optimizing individual workflows (Opt), and (3) Murakkab jointly-optimizing across workflows + multiplexing resources (Opt+Mult). 0 50 Load (RPS) Gemma-3-27B Video Q/A (High Acc.) Code Gen (High Acc.) Video Q/A (Low Lat.) Code Gen (Low Lat.) 0 20 Load (RPS) Llava-OneVision-7B 0 20 Load (RPS) Phi-4 0 25 Load (RPS) NVLM-D-72B 0 2… view at source ↗
Figure 11
Figure 11. Figure 11: Murakkab adjusts resource allocation and model instances across all workflow-SLO combinations with chang￾ing load (under a constraint of 400×H100 GPUs.) student’s solution, and (2) code generation (Figure 2b) to produce a reference solution for comparison. Scheduling. Leveraging workflow and model profiles, Mu￾rakkab identifies viable scheduling options based on clus￾ter resource availability ( [PITH_FUL… view at source ↗
Figure 12
Figure 12. Figure 12: Executing a user request involving parallel video Q/A and code generation. Murakkab selects the configuration in Figure 12c to minimize energy use while meeting the 30-second latency SLO and maintaining response quality. 20 30 60 120 180 240 360 Optimization Epoch (minutes) 0 50 100 Cost (x1000 $) Zone 1 Zone 2 Zone 3 Operational Cost Buffer Cost Utilization 0 25 50 75 100 Utilization (%) (a) Cost vs. res… view at source ↗
Figure 13
Figure 13. Figure 13: Murakkab sensitivity to optimization epoch. despite responsive demand adaptation. Frequent model and tool changes can also reduce KV cache efficiency for LLMs. Zone 2 (60–180 minutes): Balanced. Transition costs and pre￾diction uncertainty offset, yielding peak cost efficiency. Uti￾lization peaks at an epoch of around 60 minutes, reflecting the best balance between adaptation frequency and stability. Zone… view at source ↗
Figure 14
Figure 14. Figure 14: Math Q/A workflow (self-reflect [66] structure). Policy # of GPUs Energy (MWh) Cost (×1000 $) Static 4448 169.87 367.2 Mrkb Opt 1875 62.88 123.0 Mrkb Opt+Mult 1660 52.66 104.6 [PITH_FULL_IMAGE:figures/full_fig_p016_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Workflow accuracy under different configurations and the token processing load on the respective models. 0 1 # of GPUs (x1000) A100 H100 0 50 Energy (MWh) Best 78.3% Good 75.7% Fair 75.5% Basic 70.1% 0 100 Cost (x1000 $) Min Energy Min Cost (a) Accuracy SLOs. 0 1000 # of GPUs A100 H100 0 25 Energy (MWh) Best 6.5s Good 13.9s Fair 18.5s Basic 31.3s 0 200 Cost (x1000 $) Min Energy Min Cost (b) Latency SLOs … view at source ↗
Figure 16
Figure 16. Figure 16: Math Q/A workflow configured for different SLO and optimization objectives. SLO Objective Tier Model Debaters Rounds GPU TP TPOT (s) TPS Acc. Cost Best DeepSeek-Qwen-32B 4 4 A100 4 0.0767 653 Good Gemma-3-27B 4 4 A100 4 0.0624 700 Fair Gemma-3-27B 2 4 A100 4 0.0624 700 Basic Phi-4 2 4 A100 2 0.0609 623 Energy Best DeepSeek-Qwen-32B 4 4 H100 4 0.0387 1390 Good Gemma-3-27B 4 4 H100 4 0.0496 1709 Fair Gemma-… view at source ↗
Figure 17
Figure 17. Figure 17: Comparing a hand-crafted static configuration (only A100s) to: (1) Murakkab optimizing individual workflows, and (2) Murakkab jointly-optimizing across workflows + multiplexing resources (Math Q/A + Code Gen). 0 5 10 15 20 25 Time (hr) 2000 4000 Load (req/min) Chat Coding [PITH_FULL_IMAGE:figures/full_fig_p018_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Azure LLM serving traces. Ensure peak demand is met with buffer: 𝜆 peak 𝑤,𝑠 ≤ ∑︁ 𝑐∈ C𝑤,𝑚∈M 𝑥 peak 𝑤,𝑠,𝑐,𝑚 ≤ 𝛼 ·𝜆 peak 𝑤,𝑠 , ∀𝑤 ∈ W, 𝑠 ∈ S (1) Demand Satisfaction (Average): Similar bounds for average demand: 𝜆 avg 𝑤,𝑠 ≤ ∑︁ 𝑐∈ C𝑤,𝑚∈M 𝑥 avg 𝑤,𝑠,𝑐,𝑚 ≤ 𝛼 · 𝜆 avg 𝑤,𝑠 , ∀𝑤 ∈ W, 𝑠 ∈ S (2) Capacity Constraint with Multiplexing: Account for statistical multiplexing: 𝜇𝑚 · ∑︁ 𝑤,𝑠,𝑐 𝑥 peak 𝑤,𝑠,𝑐,𝑚 · 𝑡𝑐 ≤ 𝑛𝑚 · 𝜃𝑚, ∀𝑚 … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 10 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. FATE: Future-State-Aware Scheduling for Heterogeneous LLM Workflows

    cs.DC 2026-05 unverdicted novelty 7.0

    FATE reduces normalized makespan and P95 latency in real LLM workflow DAGs to 0.675 and 0.677 by jointly preserving multiple future execution states, outperforming RoundRobin by 32.5% and the strongest baseline by 8.9%.

  2. Characterizing Performance-Energy Trade-offs of Large Language Models in Multi-Request Workflows

    cs.DC 2026-03 unverdicted novelty 7.0

    This work delivers the first measurements of performance-energy trade-offs across four multi-request LLM workflow patterns on A100 GPUs using vLLM and Parrot.

  3. A Workflow-Aware Serving Layer for Agentic Applications

    cs.DC 2026-07 conditional novelty 6.5

    A workflow-aware serving layer compiles per-node model-verifier-backend plans with an ILP and adapts only uncommitted work via pre-solved pressure rungs and residual re-solves.

  4. SwarmX: Agentic Scheduling for Low-Latency Agentic Systems

    cs.DC 2026-06 unverdicted novelty 6.0

    SwarmX deploys scheduling-specific neural predictors and a scheduler-agent framework that cut tail latency by up to 61.5% and double throughput versus prior schedulers in multi-agent code, research, and multimodal workloads.

  5. SCALE: Scalable Cross-Attention Learning with Extrapolation for Agentic Workflow Scheduling

    cs.LG 2026-06 unverdicted novelty 6.0

    SCALE is a DRL scheduler using cross-attention and SRR regularization that generalizes from 16 to 48 nodes without retraining and reduces response time by 8.9% at N=48 versus the unregularized version.

  6. Natural Language Query to Configuration for Retrieval Agents

    cs.AI 2026-05 unverdicted novelty 6.0

    BRANE maps queries to optimal retrieval pipeline configurations using LLM-derived features and per-configuration correctness predictors, improving the cost-quality Pareto frontier on three benchmarks.

  7. ForkKV: Scaling Multi-LoRA Agent Serving via Copy-on-Write Disaggregated KV Cache

    cs.DC 2026-04 unverdicted novelty 6.0

    ForkKV uses copy-on-write disaggregated KV cache with DualRadixTree and ResidualAttention kernels to deliver up to 3x throughput over prior multi-LoRA serving systems with negligible quality loss.

  8. Benchmarking Compound AI Applications for Hardware-Software Co-Design

    cs.DC 2026-03 unverdicted novelty 6.0

    Introduces a benchmarking suite for compound AI applications to support cross-stack performance, cost, and resource analysis for hardware-software co-design.

  9. Rethinking Query Optimization for Multi-Agent Systems [Vision]

    cs.DB 2025-12 conditional novelty 6.0

    Agentic data pipelines are built by hand today; this paper sets a research agenda for automatically optimizing their structure, model choices, and execution engines jointly as a new query-optimization problem.

  10. SwarmX: Agentic Scheduling for Low-Latency Agentic Systems

    cs.DC 2026-06 unverdicted novelty 5.0

    SwarmX deploys scheduling-specific neural predictors and a scheduler-agent framework to reduce tail latency by up to 61.5% and double throughput in agentic AI systems on large GPU-CPU clusters.

Reference graph

Works this paper leans on

85 extracted references · 45 canonical work pages · cited by 9 Pith papers · 3 internal anchors

  1. [1]

    CrewAI — The Leading Multi-Agent Platform

    2025. CrewAI — The Leading Multi-Agent Platform. https://www. crewai.com/. Accessed: 19 August 2025

    2025

  2. [2]

    Features | Cursor – The AI Code Editor

    2025. Features | Cursor – The AI Code Editor. https://cursor.com/ features. Accessed: 19 August 2025

    2025

  3. [3]

    GitHub Copilot · Your AI pair programmer

    2025. GitHub Copilot · Your AI pair programmer. https://github.com/ features/copilot. Accessed: 19 August 2025

    2025

  4. [4]

    2025. Plix AI. https://plix.ai/. Accessed: 19 August 2025

    2025

  5. [5]

    Speaches – OpenAI-API compatible server for speech-to-text, speech-to-speech, and translation

    2025. Speaches – OpenAI-API compatible server for speech-to-text, speech-to-speech, and translation. https://speaches.ai/. Accessed: 20 August 2025

    2025

  6. [6]

    Hewett, Mojan Javaheripi, Piero Kauffmann, James R

    Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J. Hewett, Mojan Javaheripi, Piero Kauffmann, James R. Lee, Yin Tat Lee, Yuanzhi Li, Weishung Liu, Caio C. T. Mendes, Anh Nguyen, Eric Price, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Xin Wang, Rachel Ward, Yue Wu, Dingli Yu,...

    Pith/arXiv arXiv 2024

  7. [7]

    Shubham Agrawal, Adeola Adesoba, Dhruv Nandakumar, Katherine Huang, and Vignesh Srinivasakumar. 2024. Build an Agentic Video Workflow with Video Search and Summarization. NVIDIA Devel- oper Blog. Available at: https://developer.nvidia.com/blog/build-an- agentic-video-workflow-with-video-search-and-summarization/

    work page 2024

  8. [8]

    Sitaraman

    Sohaib Ahmad, Hui Guan, and Ramesh K. Sitaraman. 2024. Loki: A System for Serving ML Inference Pipelines with Hardware and Accuracy Scaling. In Proceedings of the 33rd International Symposium on High-Performance Parallel and Distributed Computing (Pisa, Italy) (HPDC ’24). Association for Computing Machinery, New York, NY, USA, 267–280. https://doi.org/10....

    arXiv 2024

  9. [9]

    Anthropic. 2025. Claude Code: Deep coding at terminal velocity. https://www.anthropic.com/claude-code. Accessed: 19 August 2025

    work page 2025

  10. [10]

    AutoGen. 2024. AutoGen. https://microsoft.github.io/autogen/stable/ /index.html

    work page 2024

  11. [11]

    Microsoft Azure. 2025. Azure AI Foundry Tool Library. https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how- to/tools/overview

    work page 2025

  12. [12]

    Microsoft Azure. 2025. Understanding costs associated with provisioned throughput units (PTU). https://learn.microsoft.com/en- us/azure/ai-foundry/openai/how-to/provisioned-throughput- onboarding

    work page 2025

  13. [13]

    Microsoft Azure. 2025. What is an AI Agent? https://learn.microsoft. com/en-us/azure/ai-foundry/agents/overview

    work page 2025

  14. [14]

    Marco Cascella, Jonathan Montomoli, Valentina Bellini, and Elena Bignami. 2023. Evaluating the feasibility of ChatGPT in healthcare: an analysis of multiple clinical and research scenarios. Journal of Medical Systems 47, 1 (2023), 33

    2023

  15. [15]

    Lingjiao Chen, Jared Quincy Davis, Boris Hanin, Peter Bailis, Ion Stoica, Matei Zaharia, and James Zou. 2024. Are More LLM Calls All You Need? Towards Scaling Laws of Compound Inference Systems. arXiv:2403.02419 [cs.LG] https://arxiv.org/abs/2403.02419

    Pith/arXiv arXiv 2024

  16. [16]

    Lingjiao Chen, Jared Quincy Davis, Boris Hanin, Peter Bailis, Matei Zaharia, James Zou, and Ion Stoica. 2025. Optimizing Model Selection for Compound AI Systems. arXiv:2502.14815 [cs.AI] https://arxiv.org/ abs/2502.14815

    Pith/arXiv arXiv 2025

  17. [17]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser, Mohammad Bavarian...

    Pith/arXiv arXiv 2021

  18. [18]

    Zhendong Chu, Shen Wang, Jian Xie, Tinghui Zhu, Yibo Yan, Jinheng Ye, Aoxiao Zhong, Xuming Hu, Jing Liang, Philip S Yu, et al . 2025. LLM agents for education: Advances and applications. arXiv preprint arXiv:2503.11733 (2025)

    arXiv 2025

  19. [19]

    Ma, Ruofan Wu, Oh Jun Kweon, Yuxuan Xia, Zhiyu Wu, and Mosharaf Chowdhury

    Jae-Won Chung, Jiachen Liu, Jeff J. Ma, Ruofan Wu, Oh Jun Kweon, Yuxuan Xia, Zhiyu Wu, and Mosharaf Chowdhury. 2025. The ML.ENERGY Benchmark: Toward Automated Inference Energy Mea- surement and Optimization. arXiv:2505.06371 [cs.LG] https://arxiv. org/abs/2505.06371

    arXiv 2025

  20. [20]

    Petar Cisar, Saša Bošnjak, and Sanja Maravic Cisar. 2010. EWMA algorithm in network practice. International Journal of Computers Communications & Control 5, 2 (2010), 160–170

    work page 2010

  21. [21]

    Google Cloud. 2025. What are AI Agents? https://cloud.google.com/ discover/what-are-ai-agents

    work page 2025

  22. [22]

    Ling Dai, Yuan-Hao Jiang, Yuanyuan Chen, Zinuo Guo, Tian-Yi Liu, and Xiaobao Shao. 2024. Agent4EDU: Advancing AI for Education with Agentic Workflows. In Proceedings of the 2024 3rd International Conference on Artificial Intelligence and Education . 180–185

    work page 2024

  23. [23]

    Databricks. 2025. Databricks Large Language Model Serving. https: //docs.databricks.com/en/large-language-models/index.html

    work page 2025

  24. [24]

    DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhi- hong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai...

    Pith/arXiv arXiv 2025

  25. [25]

    Hugging Face. 2025. Hugging Face Models. https://huggingface.co/

    work page 2025

  26. [26]

    Tao Feng, Yanzhen Shen, and Jiaxuan You. 2025. GraphRouter: A Graph-based Router for LLM Selections. arXiv:2410.03834 [cs.AI] https://arxiv.org/abs/2410.03834

    Pith/arXiv arXiv 2025

  27. [27]

    Chaoyou Fu, Yuhan Dai, Yongdong Luo, Lei Li, Shuhuai Ren, Renrui Zhang, Zihan Wang, Chenyu Zhou, Yunhang Shen, Mengdan Zhang, Peixian Chen, Yanwei Li, Shaohui Lin, Sirui Zhao, Ke Li, Tong Xu, Xi- awu Zheng, Enhong Chen, Caifeng Shan, Ran He, and Xing Sun. 2025. Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analy...

    Pith/arXiv arXiv 2025

  28. [28]

    Yao Fu, Leyang Xue, Yeqi Huang, Andrei-Octavian Brabete, Dmitrii Ustiugov, Yuvraj Patel, and Luo Mai. 2024. ServerlessLLM: Low- Latency serverless inference for large language models. In18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24). 135–153

    work page 2024

  29. [29]

    Google. 2025. Google Vertex AI Agent Garden. https://console.cloud. google.com/vertex-ai/agents/agent-garden

    work page 2025

  30. [30]

    Sagar Goyal, Eti Rastogi, Sree Prasanna Rajagopal, Dong Yuan, Fen Zhao, Jai Chintagunta, Gautam Naik, and Jeff Ward. 2024. HealAI: A healthcare LLM for effective medical documentation. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining . 1167–1168

    work page 2024

  31. [31]

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhi- nav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, et al . 2024. The Llama 3 Herd of Models. arXiv:2407.21783 [cs.CL] https://arxiv.org/abs/2407.21783

    Pith/arXiv arXiv 2024

  32. [32]

    Gurobi Optimization, LLC. 2024. Gurobi Optimizer Reference Manual. https://www.gurobi.com

    work page 2024

  33. [33]

    Jianwei Hao, Ting Jiang, Wei Wang, and In Kee Kim. 2021. An empirical analysis of VM startup times in public IaaS clouds. In 2021 IEEE 14th International Conference on Cloud Computing (CLOUD). IEEE, 398–403

    work page 2021

  34. [34]

    Junda He, Christoph Treude, and David Lo. 2025. LLM-Based Multi- Agent Systems for Software Engineering: Literature Review, Vision, and the Road Ahead. ACM Transactions on Software Engineering and Methodology 34, 5 (2025), 1–30

    work page 2025

  35. [35]

    Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021. Measuring Mathematical Problem Solving With the MATH Dataset. NeurIPS (2021)

    work page 2021

  36. [36]

    Haolin Jin, Linghan Huang, Haipeng Cai, Jun Yan, Bo Li, and Huam- ing Chen. 2024. From LLMs to LLM-based agents for software engi- neering: A survey of current, challenges and future. arXiv preprint arXiv:2408.02479 (2024)

    Pith/arXiv arXiv 2024

  37. [37]

    Yunho Jin, Gu-Yeon Wei, and David Brooks. 2025. The Energy Cost of Reasoning: Analyzing Energy Usage in LLMs with Test-time Compute. arXiv:2505.14733 [cs.LG] https://arxiv.org/abs/2505.14733

    arXiv 2025

  38. [38]

    Jiin Kim, Byeongjun Shin, Jinha Chung, and Minsoo Rhu

    work page

  39. [39]

    Gonzalez, Hao Zhang, and Ion Stoica

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica

  40. [40]

    LangChain. 2024. LangChain. https://github.com/langchain-ai/ langchain

    work page 2024

  41. [41]

    LangGraph. 2024. LangGraph. https://www.langchain.com/ langgraph

    work page 2024

  42. [42]

    Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. 2024. LLaVA-OneVision: Easy Visual Task Transfer. arXiv:2408.03326 [cs.CV] https://arxiv.org/abs/2408.03326

    Pith/arXiv arXiv 2024

  43. [43]

    Zelong Li, Shuyuan Xu, Kai Mei, Wenyue Hua, Balaji Rama, Om Raheja, Hao Wang, He Zhu, and Yongfeng Zhang. 2024. Autoflow: Automated workflow generation for large language model agents. arXiv preprint arXiv:2407.12821 (2024)

    Pith/arXiv arXiv 2024

  44. [44]

    Chaofan Lin, Zhenhua Han, Chengruidong Zhang, Yuqing Yang, Fan Yang, Chen Chen, and Lili Qiu. 2024. Parrot: efficient serving of LLM- based applications with semantic variable. In Proceedings of the 18th USENIX Conference on Operating Systems Design and Implementation (Santa Clara, CA, USA) (OSDI’24). USENIX Association, USA, Article 50, 17 pages

    work page 2024

  45. [45]

    Chunwei Liu, Matthew Russo, Michael Cafarella, Lei Cao, Peter Baile Chen, Zui Chen, Michael Franklin, Tim Kraska, Samuel Madden, Rana Shahout, and Gerardo Vitagliano. [n.d.]. Palimpzest: Optimizing AI- Powered Analytics with Declarative Query Processing. In Proceedings of the Conference on Innovative Database Research (CIDR) (2025)

    work page 2025

  46. [46]

    Chunwei Liu, Matthew Russo, Michael Cafarella, Lei Cao, Peter Baille Chen, Zui Chen, Michael Franklin, Tim Kraska, Samuel Madden, and Gerardo Vitagliano. 2024. A Declarative System for Optimizing AI Workloads. arXiv:2405.14696 [cs.CL]

    Pith/arXiv arXiv 2024

  47. [47]

    Jerry Liu. 2022. LlamaIndex. https://github.com/jerryjliu/llama_index

    2022

  48. [48]

    Jiale Liu, Yifan Zeng, Shaokun Zhang, Chi Zhang, Malte Højmark- Bertelsen, Marie Normann Gadeberg, Huazheng Wang, and Qingyun Wu. 2025. Divide, Optimize, Merge: Fine-Grained LLM Agent Opti- mization at Scale. arXiv:2505.03973 [cs.CL] https://arxiv.org/abs/2505. 03973

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  49. [49]

    Zijun Liu, Yanzhe Zhang, Peng Li, Yang Liu, and Diyi Yang. 2024. A dynamic LLM-powered agent network for task-oriented agent collab- oration. In First Conference on Language Modeling

    work page 2024

  50. [50]

    Gonzalez, and Ion Stoica

    Michael Luo, Xiaoxiang Shi, Colin Cai, Tianjun Zhang, Justin Wong, Yichuan Wang, Chi Wang, Yanping Huang, Zhifeng Chen, Joseph E. Gonzalez, and Ion Stoica. 2025. Autellix: An Efficient Serving Engine for LLM Agents as General Programs. arXiv:2502.13965 [cs.LG] https://arxiv.org/abs/2502.13965

    Pith/arXiv arXiv 2025

  51. [51]

    Xupeng Miao, Chunan Shi, Jiangfei Duan, Xiaoli Xi, Dahua Lin, Bin Cui, and Zhihao Jia. 2023. SpotServe: Serving Generative Large Lan- guage Models on Preemptible Instances. arXiv:2311.15566 [cs.DC] https://arxiv.org/abs/2311.15566

    Pith/arXiv arXiv 2023

  52. [52]

    Daye Nam, Andrew Macvean, Vincent Hellendoorn, Bogdan Vasilescu, and Brad Myers. 2024. Using an LLM to Help With Code Understand- ing. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering (Lisbon, Portugal) (ICSE ’24) . Association for Computing Machinery, New York, NY, USA, Article 97, 13 pages. https://doi.org/10.1145/359...

    arXiv 2024

  53. [53]

    Boye Niu, Yiliao Song, Kai Lian, Yifan Shen, Yu Yao, Kun Zhang, and Tongliang Liu. 2025. Flow: Modularized agentic workflow automation. arXiv preprint arXiv:2501.07834 (2025)

    Pith/arXiv arXiv 2025

  54. [54]

    Diego Novillo. 2014. SamplePGO - The Power of Profile Guided Optimizations without the Usability Burden. In 2014 LLVM Compiler Infrastructure in HPC . 22–28. https://doi.org/10.1109/LLVM-HPC. 2014.8

    work page doi:10.1109/llvm-hpc 2014

  55. [55]

    NVIDIA. 2025. NeMo Agent Toolkit. https://developer.nvidia.com/ nemo-agent-toolkit

    work page 2025

  56. [56]

    NVIDIA. 2025. NVIDIA DOCA Overview. https://docs.nvidia.com/ doca/archive/2-9-0/nvidia+doca+overview/index.html. 13

    work page 2025

  57. [57]

    Gonzalez, M Waleed Kadous, and Ion Stoica

    Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tian- hao Wu, Joseph E. Gonzalez, M Waleed Kadous, and Ion Stoica

    work page

  58. [58]

    OpenAI. 2023. ChatGPT (Mar 14 version) [Large language model]. https://chat.openai.com/chat

    work page 2023

  59. [59]

    OpenAI. 2024. Whisper Large V3 Model. https://huggingface.co/ openai/whisper-large-v3

    work page 2024

  60. [60]

    arXiv:2406.18665 [cs.LG] https://arxiv.org/abs/2406.18665

    RouteLLM: Learning to Route LLMs with Preference Data. arXiv:2406.18665 [cs.LG] https://arxiv.org/abs/2406.18665

    Pith/arXiv arXiv

  61. [61]

    OpenAI. 2025. OpenAI Agents SDK. https://openai.github.io/openai- agents-python/tools/

    work page 2025

  62. [62]

    OpenAI. 2025. OpenAI Large Language Models and API. https:// platform.openai.com/docs/

    work page 2025

  63. [63]

    OpenAI. 2025. Coding, Math, and Multimodal capabilities are the widely used benchmarks for agent and model evaluation. https: //openai.com/index/introducing-o3-and-o4-mini/

    work page 2025

  64. [64]

    Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. 2023. ToolLLM: Facilitating large language models to master 16000+ real-world APIs. arXiv preprint arXiv:2307.16789 (2023)

    Pith/arXiv arXiv 2023

  65. [65]

    Deepti Raghavan, Keshav Santhanam, Muhammad Shahir Rahman, Nayani Modugula, Luis Gaspar Schroeder, Maximilien Cura, Houjun Liu, Pratiksha Thaker, Philip Levis, and Matei Zaharia. 2025. Alto: Or- chestrating Distributed Compound AI Systems with Nested Ancestry. arXiv:2403.04311 [cs.AI] https://arxiv.org/abs/2403.04311

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  66. [66]

    Model Context Protocol. 2025. Model Context Protocol (MCP). https: //modelcontextprotocol.io/docs/getting-started/intro

    work page 2025

  67. [67]

    Francisco Romero, Johann Hauswald, Aditi Partap, Daniel Kang, Matei Zaharia, and Christos Kozyrakis. 2022. Optimizing Video Analytics with Declarative Model Relationships. Proc. VLDB Endow. 16, 3 (Nov. 2022), 447–460. https://doi.org/10.14778/3570690.3570695

    work page arXiv 2022

  68. [68]

    Llama: A Heterogeneous & Serverless Framework for Auto-Tuning Video Analytics Pipelines

    Francisco Romero, Mark Zhao, Neeraja J. Yadwadkar, and Christos Kozyrakis. 2021. Llama: A Heterogeneous & Serverless Framework for Auto-Tuning Video Analytics Pipelines. arXiv:2102.01887 [cs.DC] https://arxiv.org/abs/2102.01887

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  69. [69]

    Matthew Renze and Erhan Guven. 2024. Self-reflection in llm agents: Effects on problem-solving performance. arXiv preprint arXiv:2405.06682 (2024)

    Pith/arXiv arXiv 2024

  70. [70]

    Jovan Stojkovic, Chaojie Zhang, Íñigo Goiri, Josep Torrellas, and Esha Choukse. 2025. DynamoLLM: Designing LLM inference clusters for performance and energy efficiency. In 2025 IEEE International Sym- posium on High Performance Computer Architecture (HPCA) . IEEE, 1348–1362

    work page 2025

  71. [71]

    Xin Tan, Yimin Jiang, Yitao Yang, and Hong Xu. 2025. Teola: Towards End-to-End Optimization of LLM-based Applications. arXiv:2407.00326 [cs.DC] https://arxiv.org/abs/2407.00326

    arXiv 2025

  72. [72]

    Stanford NLP Group. 2023. DSPy: The Framework for Pro- gramming—Not Prompting—Language Models. https://github.com/ stanfordnlp/dspy

    work page 2023

  73. [73]

    Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexan- dre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Ge- offrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penchev, Gaël Liu, Francesco Visin, Kathleen Kenealy, Lucas...

    Pith/arXiv arXiv 2025

  74. [74]

    Jize Wang, Ma Zerun, Yining Li, Songyang Zhang, Cailian Chen, Kai Chen, and Xinyi Le. 2024. GTA: a benchmark for general tool agents. Advances in Neural Information Processing Systems 37 (2024), 75749– 75790

    work page 2024

  75. [75]

    Yunlong Tang, Jing Bi, Siting Xu, Luchuan Song, Susan Liang, Teng Wang, Daoan Zhang, Jie An, Jingyang Lin, Rongyi Zhu, Ali Vosoughi, Chao Huang, Zeliang Zhang, Pinxin Liu, Mingqian Feng, Feng Zheng, Jianguo Zhang, Ping Luo, Jiebo Luo, and Chenliang Xu

  76. [76]

    arXiv:2312.17432 [cs.CV] https://arxiv.org/abs/2312.17432

    Video Understanding with Large Language Models: A Survey. arXiv:2312.17432 [cs.CV] https://arxiv.org/abs/2312.17432

    arXiv

  77. [77]

    Shirley Wu, Parth Sarthi, Shiyu Zhao, Aaron Lee, Herumb Shandilya, Adrian Mladenic Grobelnik, Nurendra Choudhary, Eddie Huang, Karthik Subbian, Linjun Zhang, et al . 2025. Optimas: Optimizing Compound AI Systems with Globally Aligned Local Rewards. arXiv preprint arXiv:2507.03041 (2025)

    arXiv 2025

  78. [78]

    Jiayi Zhang, Jinyu Xiang, Zhaoyang Yu, Fengwei Teng, Xionghui Chen, Jiaqi Chen, Mingchen Zhuge, Xin Cheng, Sirui Hong, Jinlin Wang, et al. 2024. Aflow: Automating agentic workflow generation. arXiv preprint arXiv:2410.10762 (2024)

    Pith/arXiv arXiv 2024

  79. [79]

    Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. 2024. Executable Code Actions Elicit Better LLM Agents. In Forty-first International Conference on Machine Learning . https://openreview.net/forum?id=jJ9BoXAfFa

    work page 2024

  80. [80]

    Patrick Wintermeyer, Maria Apostolaki, Alexander Dietmüller, and Laurent Vanbever. 2020. P2GO: P4 profile-guided optimizations. In Proceedings of the 19th ACM Workshop on Hot Topics in Networks . 146– 152

    work page 2020

Showing first 80 references.

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.