Pith. sign in

REVIEW 4 major objections 5 minor 49 references

SpecBox claims that sandbox preparation for LLM tool calls can be hidden behind token generation and predicted across steps, cutting P99 end-to-end latency by up to 2.9x and peak memory by 45.9% compared with on-demand and permanently reser

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-07-31 23:28 UTC pith:DLT44XMK

load-bearing objection A well-engineered prewarming system for LLM agents, but the semantic cache can return wrong results and the evaluation is largely in-sample, so the headline numbers should not be taken at face value yet. the 4 major comments →

arxiv 2607.23933 v1 pith:DLT44XMK submitted 2026-07-27 cs.DC cs.AIcs.LGcs.PF

SpecBox: Speculative Sandbox Scheduling for Efficient LLM Agent Serving

Yihui Zhang (1) , Tianyu Wo (1) , Jinghao Wang (1) , Xiaoyang Sun (2) , Menghao Zhang (1) , Cangzhou Yuan (1) , Li Li (1) , Chunming Hu (1)
show 4 more authors
Albert Y. Zomaya (3) Renyu Yang (1) ((1) Beihang University (2) University of Leeds (3) The University of Sydney)
This is my paper
classification cs.DC cs.AIcs.LGcs.PF
keywords LLM agentssandbox cold startspeculative prewarmingMarkov prefetchingsemantic cachingout-of-band data planeserverless latencyModel Context Protocol
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper is trying to show that the cold-start cost of isolated tool sandboxes does not have to sit on the critical path of an LLM agent. Its runtime, SpecBox, watches the model's streaming token output and prewarms the sandbox it believes the agent will call next, then uses historical tool-transition patterns to push that prediction one step further ahead. Two complementary optimizations remove the remaining overhead: a semantic cache that reuses results of equivalent earlier calls, and an out-of-band shared-memory path that moves large artifacts without serializing them through the control channel. If the argument holds, an agent-serving platform can deliver close to 'keep everything warm' latency while paying close to 'start on demand' memory costs. The measured claims are up to 2.9x lower 99th-percentile end-to-end latency than on-demand serving and 45.9% lower peak memory than permanently reserved sandboxes.

Core claim

SpecBox's central claim is that the sequential dependency chain of an agent loop—LLM generates tokens, then sandbox starts, then tool runs—is not unavoidable. It begins preparing a sandbox as soon as either a keyword router or a semantic router detects a likely tool intent in the partially generated token stream, overlapping environment setup with the remaining inference time. To cover sandboxes whose cold starts exceed that overlap window, it maintains a sandbox dependency graph with first-order Markov transition probabilities learned from execution traces and prewarms the top candidates within a fixed budget before the next step commits. Once execution is ready, a semantic result cache and

What carries the argument

The load-bearing mechanism is speculative preallocation, implemented as two coupled predictors: an in-step intent router that unions keyword and semantic triggers on the streaming token output, and a cross-step stochastic prefetcher that runs a first-order Markov model over a sandbox dependency graph (a directed graph of observed tool-to-tool transitions with Laplace-smoothed edge probabilities) and selects the top-B candidates above a cost and probability threshold. Complementing these are a semantic result cache—reuse only when tool identity matches and embedding similarity exceeds a threshold—and an out-of-band shared-memory transport that carries large artifacts by reference rather than

Load-bearing premise

The central premise—that the next sandbox an agent needs is predictable from a first-order Markov model over historical tool transitions, and that the 200 replayed sessions fairly represent that history—is what the prefetch gains rest on; if those transition patterns are unrepresentative or change over time, the reported speedups shrink.

What would settle it

Run a train/test split: build the sandbox dependency graph from half of the 200 traced sessions, replay the other half, and compare cross-step prefetch hit rate and per-turn latency against the in-sample numbers; if the hit rate approaches random or the 6.0x per-turn gain vanishes, the prefetch result was overfit to the traced workload.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Tool sandbox provisioning can be moved out of the agent's critical path in most steps: cumulative provisioning latency drops 4.53x and comes within 10.6% of a permanently reserved deployment.
  • Cross-step prefetching is where most of the multi-turn gain lives: by turn 10 the proactive variant cuts per-turn wait from about 540 ms to about 97 ms relative to the reactive-only variant, keeping cold starts below one per turn.
  • Semantically equivalent but differently worded tool calls can be served from cache: average wait drops 2.91x over no cache, with a 37.4% hit rate on the repeated-request workload.
  • Delivering large tool artifacts over a shared-memory reference keeps data-transfer latency nearly flat as payloads grow, reaching about 6 ms at 1 GB versus about 1.87 s for inline JSON-RPC.
  • These mechanisms sit in a middleware layer and preserve the tool-facing interface, so existing agent frameworks can adopt the runtime without changing how tools are invoked.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The first-order Markov assumption is the point most likely to limit generalization: if a deployment's tool transitions drift over time or depend on long-horizon plans, the SDG probabilities could go stale; testing on a disjoint train/test split of traces would show how much of the 6.0x cross-step gain is real.
  • The same overlap principle transfers to agentic reinforcement learning, where per-rollout environment resets resemble sandbox cold starts; the paper notes this direction but does not evaluate it.
  • Semantic caching is only safe for deterministic tools; a production extension would attach per-tool determinism and side-effect annotations and route non-deterministic tools around the cache automatically.
  • The prefetch budget B=1 is conservative; if memory pressure allows, dynamically adjusting B with current cold-start cost and cache state could hide more cold starts without reverting to fully reserved deployments.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper presents SpecBox, a runtime middleware for LLM agent serving that overlays speculative scheduling on top of sandboxed MCP-style tool execution. It contributes three mechanisms: (i) intent-aware sandbox prewarming that uses keyword and semantic routers on streaming token output to overlap sandbox initialization with LLM decoding; (ii) stochastic sandbox prefetching based on a first-order Markov model over a sandbox dependency graph (SDG); and (iii) reuse-aware data transmission consisting of a semantic result cache and an out-of-band shared-memory data plane. The prototype is integrated with AgentScope and evaluated on 200 MCPBench-derived multi-turn traces using Qwen3.5-Max. The paper reports a 2.9× P99 end-to-end latency improvement over an on-demand baseline and a 45.9% peak-memory reduction versus a reserved-sandbox baseline, with microbenchmarks attributing the gains to each mechanism. The central idea is plausible and the system decomposition is clear, but the evaluation currently does not support the generality of the reported numbers and the semantic cache's correctness guarantee is not demonstrated.

Significance. If the claims hold, SpecBox would be a useful contribution: it identifies a concrete opportunity to hide sandbox cold starts inside LLM token generation, extends prewarming across steps with a lightweight stochastic model, and separates large artifact transfer from the control plane. The authors also provide mechanism-level microbenchmarks, which is a strength. However, the headline latency and memory results are obtained under a single-seed, same-benchmark evaluation with no visible train/test split for the predictive components, and the semantic-result-cache mechanism raises an unverified correctness risk. The contribution is significant but the evidence is currently insufficient as presented.

major comments (4)
  1. [§5.1, §3.2 (Eq. 4), Fig. 11] The stochastic prefetcher estimates transition probabilities from historical traces and updates them online, but the evaluation replays the same 200 generated sessions with a fixed seed (seed = 0). No train/test split is described for the SDG counts. The reported 6.0× cross-step latency reduction and the E2E speedups are therefore in-sample estimates: the predictor may be memorizing the replayed transitions rather than generalizing. Please add a held-out or leave-one-session-out evaluation, state how many sessions populate the SDG before measurement, and report prediction accuracy on unseen traces.
  2. [§5.3, §5.1] All decisive operating points (γ=2, τ=0.6, B=1, λ=5, τ_c=0.8) are either explicitly selected from the §5.3 sensitivity experiments or reported only for the same workload, and the E2E evaluation uses one fixed seed. This means the headline numbers are tuned on the test set. Please provide a clear separation of tuning/validation/test traces, or otherwise demonstrate that the thresholds are not selected using the same sessions that produce the reported P99 and memory results. Reporting variance over multiple seeds is also needed.
  3. [§3.3, Eq. (8); §4; §5.3.3] The semantic cache's reuse condition is only tool(x)=tool(x_i) and sim(φ(x),φ(x_i))≥τ_c. This does not establish functional equivalence: for a deterministic tool, invocations with different arguments, query filters, or file paths can have high embedding similarity yet produce different outputs and side effects. The paper calls this a 'conservative extension of exact reuse' and mentions a validation fallback, but no concrete validation mechanism or formal equivalence condition is given, and the trace replay does not compare cached results against freshly computed outputs. Challenge #3 is therefore not demonstrated, and part of the reported speedup may come from skipped executions that should have run. Please add an explicit correctness check, describe the fallback path, and report output-equality verification on the evaluation traces.
  4. [Table 4 vs. §4] Table 4 labels the semantic cache experiments as 'Semantic Cache (τ=0.6)', while §4 states that τ_c=0.8 was used in the reported experiments. The 2.91× cache speedup and the 84.8% bypass ratio are thus not tied to the stated production configuration, and the discrepancy is load-bearing for the claimed benefit of semantic caching. Please clarify which threshold was actually used and present results for the relevant threshold range, including the relationship between τ_c, hit rate, and correctness.
minor comments (5)
  1. [Figures 9 and 10, §5.2] The system is named SpecBox, but several figures and the text in §5.2.2 refer to 'Laplace' or 'Laplace'. This naming inconsistency should be fixed throughout.
  2. [Table 4] The column 'Bypass Ratio' is ambiguous: it could mean the fraction of cache hits that skip execution, or the fraction that are bypassed for validation. Its meaning should be defined, especially because Exact-Match Cache has a 100.0% value.
  3. [§5.3.3] Typo: 'sematic caching' should be 'semantic caching'.
  4. [§8 / Abstract] The conclusion mentions a '97.9% prewarming hit rate' that is not defined or measured in the evaluation section. Please either report the corresponding measurement and definition or remove the claim.
  5. [§5.1] The workload generation uses an LLM planner and external APIs, but no artifact or code release is mentioned. For reproducibility, specify the exact planner prompts, model version, date of access, and dataset release plan.

Circularity Check

1 steps flagged

Stochastic sandbox prefetching is fitted to the same 200 traces that are replayed in evaluation, making its 6.0x/97.9% predictive gains in-sample; the rest of SpecBox is empirical rather than circular.

specific steps
  1. fitted input called prediction [§3.2 (Eqs. 3-4, Online Update) with §5.1 Methodology and §5.3.2 / §8]
    "For each ordered pair (v_i,v_j), we maintain transition counts: C_{i,j}←C_{i,j}+1 ... (3). The next-state probability is estimated by a first-order Markov model with Laplace Smoothing: P_{i,j}=... (4). ... After each completed sandbox invocation, SpecBox appends one transition edge to SDG and updates C_{i,j} and P_{i,j} asynchronously ... Unless otherwise specified, we evaluate all systems by replaying full traces as session-level workloads with a fixed random seed (seed = 0)."

    The Markov transition probabilities are empirical counts over the very workload used for evaluation. Since §5.1 replays the same 200 trajectories and the online-update paragraph updates C and P during execution, the SDG encodes the test set's own transition frequencies before and while it is being measured. The 'prediction' of the next sandbox is therefore an in-sample majority/empirical-frequency estimate, not an independent forecast. The 6.0x cross-step latency reduction (Fig. 11) and the 97.9% prewarming hit rate (Conclusion) then measure how repetitive the test traces are, not generalization to unseen agent workflows. Routing thresholds (gamma, tau, tau_c) are also chosen on this same benchmark (§5.3), compounding the in-sample selection.

full rationale

SpecBox is an empirical serving system, not a formal derivation, so most of its reported speedups are measurements rather than first-principles predictions. I found no load-bearing self-citation chain: the cited prior work (AgentScope, MCPBench, Cauchy) is used as a framework/benchmark and is not invoked to forbid alternatives or justify the core mechanism. The semantic cache (Eq. 8) is a correctness risk because embedding similarity is asserted, not proven, to imply functional equivalence; that is an omitted proof, not a circular reduction, and I did not score it here. The one structural circularity is the stochastic prefetcher: its probability model is trained on the identical traces that are replayed as the test workload, with no described train/test split, and the headline prefetching gains are in-sample fits. Because this affects one of the three central mechanisms and the main E2E result aggregates that mechanism, the score is 6 rather than 0-2. The paper's own limitation (first-order Markov may be inaccurate in open-ended workflows) acknowledges predictive weakness but does not repair the in-sample evaluation.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 0 invented entities

The central claim rests on multiple fitted thresholds and on domain assumptions about the predictability of agent tool use. The Markov model and the semantic-equivalence proxy are the most significant assumptions; both are tuned or validated only on the paper's own 200-trace benchmark.

free parameters (6)
  • γ (keyword trigger threshold) = 2
    Chosen in §5.3.1 from sensitivity analysis on the same benchmark (γ=1,2,3); affects prewarm timing and false-positive rate.
  • τ (prefetch probability threshold) = 0.6
    Configured in §4; filters low-confidence successor sandboxes in Eq. 6.
  • B (prefetch budget) = 1
    Limits number of sandboxes prewarmed per step; set to 1 in §4.
  • λ (cold-start cost threshold) = 5
    Cost threshold in Eq. 5; set in §4.
  • τ_c (semantic cache similarity threshold) = 0.8
    Used in Eq. 8; set in §4 and used in reported experiments.
  • Semantic router model = Retrieval (TF-IDF)
    Selected in Table 2 as the best operating point on the benchmark; a model-selection choice.
axioms (5)
  • ad hoc to paper Agent tool transitions are well-modeled by a first-order Markov chain over sandbox states
    Introduced in §3.2; not justified by data or theory. If tool choices depend on longer context, the SDG-based prefetch set will be wrong.
  • domain assumption Tool invocations can be classified as deterministic for safe semantic caching
    §3.3 restricts caching to deterministic invocations; requires tools to be side-effect-free and detectably so.
  • domain assumption LLM serving frameworks expose streaming token outputs at runtime
    Required for the Keyword/Semantic routers; depends on integration with frameworks like AgentScope.
  • ad hoc to paper Semantically similar normalized invocations imply functionally equivalent results
    Eq. 8 uses similarity threshold τ_c as a proxy for equivalence; no verification is provided for arbitrary tools.
  • domain assumption Sandbox cold-start latency is dominated by application-layer handshakes rather than OS boot, making prewarm effective
    §2.2/2.3 claims image/namespace setup and handshakes cost seconds; if sandbox start is already sub-millisecond, overlap produces little benefit.

pith-pipeline@v1.3.0-alltime-deepseek · 19154 in / 12221 out tokens · 120968 ms · 2026-07-31T23:28:08.632620+00:00 · methodology

0 comments
read the original abstract

As LLM agents increasingly rely on the Model Context Protocol (MCP) to invoke isolated external sandboxes, disaggregated sandbox deployment introduces a fundamental tension between resource utilization and interactive tail latency. Persistent long-lived sandbox reservations incur excessive memory overhead at scale, while lazy on-demand instantiation generates severe cold-start penalties that degrade response performance under multi-tenant, multi-turn agent workloads. To resolve this dilemma, we present SpecBox, a runtime built around speculative sandbox preallocation tailored for dynamic LLM agent execution pipelines. At its core, SpecBox implements keyword matching and streaming semantic embedding to enable intent-driven sandbox prewarming, which identifies pending tool execution demands mid-LLM token generation and fully overlaps sandbox bootstrapping with model inference. To extend prewarming windows across sequential agent steps, the framework leverages context-aware stochastic prefetching atop a sandbox dependency graph to probabilistically forecast future sandbox switches ahead of execution. We complement these speculative mechanisms with two orthogonal optimizations: a semantic result cache that prunes redundant repeated sandbox invocations, and a dedicated out-of-band shared-memory transport plane that bypasses conventional network serialization to deliver zero-copy artifact transfers. Evaluated on high-concurrency multi-turn agent traces, our prototype demonstrates that SpecBox cuts P99 end-to-end latency by up to $2.9\times$ relative to the on-demand sandbox baseline, while slashing peak memory consumption by $45.9\%$ compared to permanently reserved sandbox deployments.

Figures

Figures reproduced from arXiv: 2607.23933 by (2) University of Leeds, (3) The University of Sydney), Albert Y. Zomaya (3), Cangzhou Yuan (1), Chunming Hu (1), Jinghao Wang (1), Li Li (1), Menghao Zhang (1), Renyu Yang (1) ((1) Beihang University, Tianyu Wo (1), Xiaoyang Sun (2), Yihui Zhang (1).

Figure 1
Figure 1. Figure 1: Proactive and overlapped execution v.s. the native vanilla approaches [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Execution bottlenecks in agent runtimes. (a) Execution-step latency [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Intent-aware sandbox prewarming: the plan agent expands user requests into ReAct steps, and keyword plus semantic predictions jointly decide to [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Tradeoffs among routing policies. Keyword routing trades early [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Stochastic sandbox prefetching in SpecBox. Execution history predicts likely sandboxes for subsequent ReAct steps. without compromising the timeliness of prewarming. This is done by streaming context to two independent routers. As shown in [PITH_FULL_IMAGE:figures/full_fig_p005_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Stochastic sandbox prefetching using a first-order Markov model. [PITH_FULL_IMAGE:figures/full_fig_p005_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Reuse-aware data transmission in SpecBox. A semantic-cache hit re￾turns a prior result (paper_search); a miss executes the sandbox (paper_slides) and delivers the result through the out-of-band data path. 3.3 Reuse-Aware Data Transmission Once an invocation is ready, its remaining critical-path cost lies in artifact movement (𝑇 (𝑁 ) 𝑑𝑎𝑡𝑎_𝑖𝑜 ) and tool execution (𝑇 (𝑁 ) 𝑠𝑎𝑛𝑑𝑏𝑜𝑥_𝑒𝑥𝑒𝑐 ). SpecBox first checks … view at source ↗
Figure 8
Figure 8. Figure 8: Overview of SpecBox. standard path and appends the fully computed result to the cache. On a cache hit, both sandbox initialization and tool invocation are skipped, thereby reducing 𝑇 (𝑁 ) 𝑒𝑛𝑣_𝑝𝑟𝑒𝑝 and 𝑇 (𝑁 ) 𝑠𝑎𝑛𝑑𝑏𝑜𝑥_𝑒𝑥𝑒𝑐 . As will be demonstrated in § 5.3.3, the semantic caching strategy can recover a larger proportion of redundant computation than using the strategy of exact matching alone, while maintain… view at source ↗
Figure 9
Figure 9. Figure 9: Cumulative sandbox provisioning latency across multi-turn agent [PITH_FULL_IMAGE:figures/full_fig_p008_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: End-to-end performance and resource consumption under concurrent workloads: (a) P99 E2E latency, (b) mean cumulative sandbox provisioning latency, [PITH_FULL_IMAGE:figures/full_fig_p009_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Dynamic execution profiling across a 10-turn conversation horizon, [PITH_FULL_IMAGE:figures/full_fig_p010_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Data transmission latency scaling profiles under varying payload [PITH_FULL_IMAGE:figures/full_fig_p011_12.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

49 extracted references · 6 linked inside Pith

  1. [1]

    Alexandru Agache, Marc Brooker, Alexandra Iordache, Anthony Liguori, Rolf Neugebauer, Phil Piwonka, and Diana-Maria Popa. 2020. Firecracker: Lightweight virtualization for serverless applications. In17th USENIX symposium on networked systems design and implementation (NSDI 20). 419–434

    2020

  2. [2]

    Amey Agrawal, Nitin Kedia, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav Gulavani, Alexey Tumanov, and Ramachandran Ramjee. 2024. Taming {Throughput-Latency} tradeoff in{LLM} inference with{Sarathi-Serve}. In 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24). 117–134

    2024

  3. [3]

    2026.DashScope - Alibaba Cloud: AI and Cloud Computing Services

    Alibaba Cloud. 2026.DashScope - Alibaba Cloud: AI and Cloud Computing Services. https://dashscope.console.aliyun.com/ Retrieved June, 2026. 12 SpecBox: Speculative Sandbox Scheduling for Efficient LLM Agent Serving Conference’17, July 2017, Washington, DC, USA

    2026

  4. [4]

    2026.Amazon Bedrock AgentCore- A WS

    Amazon Web Services. 2026.Amazon Bedrock AgentCore- A WS. https://aws. amazon.com/bedrock/agentcore/ Retrieved June, 2026

    2026

  5. [5]

    2024.Introducing the Model Context Protocol

    Anthropic PBC. 2024.Introducing the Model Context Protocol. https://www. anthropic.com/news/model-context-protocol Retrieved June, 2026

    2024

  6. [6]

    Lixiang Ao, George Porter, and Geoffrey M Voelker. 2022. Faasnap: Faas made fast using snapshot-based vms. InProceedings of the Seventeenth European Conference on Computer Systems. 730–746

    2022

  7. [7]

    2026.Jupyter MCP Server

    Datalayer. 2026.Jupyter MCP Server. https://github.com/datalayer/jupyter-mcp- server Retrieved June, 2026

    2026

  8. [8]

    Dong Du, Tianyi Yu, Yubin Xia, Binyu Zang, Guanglu Yan, Chenggang Qin, Qix- uan Wu, and Haibo Chen. 2020. Catalyzer: Sub-millisecond startup for serverless computing with initialization-less booting. InProceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems. 467–481

    2020

  9. [9]

    Dawei Gao, Zitao Li, Xuchen Pan, Weirui Kuang, Zhijian Ma, Bingchen Qian, Fei Wei, Wenhao Zhang, Yuexiang Xie, Daoyuan Chen, et al. 2024. Agentscope: A flexible yet robust multi-agent platform.arXiv preprint arXiv:2402.14034(2024)

    Pith/arXiv arXiv 2024

  10. [10]

    2026.MCP Registry

    GitHub. 2026.MCP Registry. https://github.com/mcp Retrieved June, 2026

    2026

  11. [11]

    2026.Gemini Enterprise Agent Platform

    Google Cloud. 2026.Gemini Enterprise Agent Platform. https://docs.cloud.google. com/gemini-enterprise-agent-platform/scale/sandbox Retrieved June, 2026

    2026

  12. [12]

    Jialiang Huang, Teng Ma, Zheng Liu, Sixing Lin, Kang Chen, Jinlei Jiang, Xia Liao, Yingdi Shan, Yongwei Wu, Ning Zhang, Mengting Lu, Tao Ma, Haifeng Gong, and Mingxing Zhang. 2026. TrEnv-X: Transparently Share Serverless Execution Environments Across Different Functions and Nodes.ACM Trans. Comput. Syst. (March 2026). https://doi.org/10.1145/3805475 Just Accepted

    doi:10.1145/3805475 2026

  13. [13]

    Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2017. Bag of Tricks for Efficient Text Classification. InProceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers. Association for Computational Linguistics, 427–431

    2017

  14. [14]

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th Symposium on Operating Systems Principles. 611–626

    2023

  15. [15]

    2026.LangGraph: Agent Orchestration Framework for Reliable AI Agents

    Langchain-ai. 2026.LangGraph: Agent Orchestration Framework for Reliable AI Agents. https://www.langchain.com/langgraph Retrieved June, 2026

    2026

  16. [16]

    1812.Théorie analytique des probabilités

    Pierre-Simon Laplace. 1812.Théorie analytique des probabilités. Courcier, Paris. http://gallica.bnf.fr/ark:/12148/bpt6k88764q

  17. [17]

    Tianyu Li, Badrish Chandramouli, Philip A Bernstein, and Samuel Madden. 2024. Distributed Speculative Execution for Resilient Cloud Applications.arXiv preprint arXiv:2412.13314(2024)

    Pith/arXiv arXiv 2024

  18. [18]

    Xinyi Li, Sai Wang, Siqi Zeng, Yu Wu, and Yi Yang. 2024. A survey on LLM-based multi-agent systems: workflow, infrastructure, and challenges.Vicinagearth1, 1 (2024), 9

    2024

  19. [19]

    2022.{RunD}: A lightweight secure container runtime for high-density deployment and high-concurrency startup in serverless computing

    Zijun Li, Jiagan Cheng, Quan Chen, Eryu Guan, Zizheng Bian, Yi Tao, Bin Zha, Qiang Wang, Weidong Han, and Minyi Guo. 2022.{RunD}: A lightweight secure container runtime for high-density deployment and high-concurrency startup in serverless computing. In2022 USENIX Annual Technical Conference (USENIX ATC 22). 53–68

    2022

  20. [20]

    Zijun Li, Chuhao Xu, Quan Chen, Jieru Zhao, Chen Chen, and Minyi Guo. 2023. Dataflower: Exploiting the data-flow paradigm for serverless workflow orches- tration. InProceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 4. 57–72

    2023

  21. [21]

    David H Liu, Amit Levy, Shadi Noghabi, and Sebastian Burckhardt. 2023. Doing more with less: Orchestrating serverless applications without an orchestrator. In 20th USENIX symposium on networked systems design and implementation (NSDI 23). 1505–1519

    2023

  22. [22]

    Fangming Lu, Xingda Wei, Zhuobin Huang, Rong Chen, Minyu Wu, and Haibo Chen. 2024. Serialization/deserialization-free state transfer in serverless work- flows. InProceedings of the Nineteenth European Conference on Computer Systems. 132–147

    2024

  23. [23]

    Ashraf Mahgoub, Edgardo Barsallo Yi, Karthick Shankar, Sameh Elnikety, Somali Chaterji, and Saurabh Bagchi. 2022. {ORION} and the three rights: Sizing, bundling, and prewarming for serverless{DAGs}. In16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22). 303–320

    2022

  24. [24]

    Kai Mei, Xi Zhu, Wujiang Xu, Mingyu Jin, Wenyue Hua, Zelong Li, Shuyuan Xu, Ruosong Ye, Yingqiang Ge, and Yongfeng Zhang. [n. d.]. AIOS: LLM Agent Operating System. InSecond Conference on Language Modeling

  25. [25]

    2026.Azure Container Apps Sandboxes

    Microsoft. 2026.Azure Container Apps Sandboxes. https://sandboxes.azure.com/ Retrieved June, 2026

    2026

  26. [26]

    2026.Playwright MCP

    Microsoft. 2026.Playwright MCP. https://github.com/microsoft/playwright# playwright-mcp Retrieved June, 2026

    2026

  27. [27]

    2026.Neo4j official MCP Server

    Neo4j. 2026.Neo4j official MCP Server. https://github.com/neo4j/mcp Retrieved June, 2026

    2026

  28. [28]

    OpenAGS. 2026.openags/paper-search-mcp: MCP, CLI, Skills for searching and downloading academic papers from multiple sources like arXiv, PubMed, bioRxiv, etc.https://github.com/openags/paper-search-mcp Retrieved June, 2026

    2026

  29. [29]

    Carlo Puliafito, Claudio Cicconetti, Marco Conti, Enzo Mingozzi, and Andrea Pas- sarella. 2022. Stateless or stateful FaaS? I’ll take both!. In2022 IEEE International Conference on Smart Computing (SMARTCOMP). IEEE, 62–69

    2022

  30. [30]

    Rohan Basu Roy, Tirthak Patel, and Devesh Tiwari. 2022. Icebreaker: Warming serverless functions better with heterogeneity. InProceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems. 753–767

    2022

  31. [31]

    2026.all-MiniLM-L6-v2

    sentence-transformers. 2026.all-MiniLM-L6-v2. https://huggingface.co/sentence- transformers/all-MiniLM-L6-v2 Retrieved June, 2026

    2026

  32. [32]

    Mohammad Shahrad, Rodrigo Fonseca, Inigo Goiri, Gohar Chaudhry, Paul Ba- tum, Jason Cooke, Eduardo Laureano, Colby Tresness, Mark Russinovich, and Ricardo Bianchini. 2020. Serverless in the wild: Characterizing and optimizing the serverless workload at a large cloud provider. In2020 USENIX annual technical conference (USENIX ATC 20). 205–218

    2020

  33. [33]

    Guangming Sheng, Chi Zhang, Zilingfeng Ye, Xibin Wu, Wang Zhang, Ru Zhang, Yanghua Peng, Haibin Lin, and Chuan Wu. 2024. HybridFlow: A Flexible and Efficient RLHF Framework.arXiv preprint arXiv: 2409.19256(2024)

    Pith/arXiv arXiv 2024

  34. [34]

    Karen Sparck Jones. 1972. A statistical interpretation of term specificity and its application in retrieval.Journal of documentation28, 1 (1972), 11–21

    1972

  35. [35]

    Jovan Stojkovic, Tianyin Xu, Hubertus Franke, and Josep Torrellas. 2023. Mxfaas: Resource sharing in serverless environments for parallelism and efficiency. In Proceedings of the 50th annual international symposium on computer architecture. 1–15

    2023

  36. [36]

    Jovan Stojkovic, Tianyin Xu, Hubertus Franke, and Josep Torrellas. 2023. Spec- faas: Accelerating serverless applications with speculative function execution. In 2023 IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 814–827

    2023

  37. [37]

    Yifan Sui, Hanfei Yu, Yitao Hu, Jianxun Li, and Hao Wang. 2024. Pre-warming is not enough: Accelerating serverless inference with opportunistic pre-loading. In Proceedings of the 2024 ACM Symposium on Cloud Computing. 178–195

    2024

  38. [38]

    Xin Tan, Yimin Jiang, Yitao Yang, and Hong Xu. 2025. Towards end-to-end optimization of llm-based applications with ayo. InProceedings of the 30th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2. 1302–1316

    2025

  39. [39]

    Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. 2024. A survey on large language model based autonomous agents.Frontiers of Computer Science18, 6 (2024), 186345

    2024

  40. [40]

    Zhenting Wang, Qi Chang, Hemani Patel, Shashank Biju, Cheng-En Wu, Quan Liu, Aolin Ding, Alireza Rezazadeh, Ankit Shah, Yujia Bao, et al . 2025. Mcp- bench: Benchmarking tool-using llm agents with complex real-world tasks via mcp servers.arXiv preprint arXiv:2508.20453(2025)

    Pith/arXiv arXiv 2025

  41. [41]

    2026.WebAssembly

    WebAssembly. 2026.WebAssembly. https://webassembly.org/ Retrieved June, 2026

    2026

  42. [42]

    Xingda Wei, Fangming Lu, Tianxia Wang, Jinyu Gu, Yuhan Yang, Rong Chen, and Haibo Chen. 2023. No provisioned concurrency: Fast{RDMA-codesigned} remote fork for serverless computing. In17th USENIX Symposium on Operating Systems Design and Implementation (OSDI 23). 497–517

    2023

  43. [43]

    Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, et al . 2023. Autogen: En- abling next-gen llm applications via multi-agent conversation.arXiv preprint arXiv:2308.08155(2023)

    Pith/arXiv arXiv 2023

  44. [44]

    Chuhao Xu, Yiyu Liu, Zijun Li, Quan Chen, Han Zhao, Deze Zeng, Qian Peng, Xueqi Wu, Haifeng Zhao, Senbo Fu, et al. 2024. Faasmem: Improving memory efficiency of serverless computing with memory pool architecture. InProceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3. 331–348

    2024

  45. [45]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629(2022)

    Pith/arXiv arXiv 2022

  46. [46]

    Hanfei Yu, Rohan Basu Roy, Christian Fontenot, Devesh Tiwari, Jian Li, Hong Zhang, Hao Wang, and Seung-Jong Park. 2024. Rainbowcake: Mitigating cold- starts in serverless with layer-wise container caching and sharing. InProceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 1. 335–350

    2024

  47. [47]

    Yihui Zhang, Han Shen, Renyu Yang, Di Tian, Yuxi Luo, Menghao Zhang, Li Li, Chunming Hu, Tianyu Wo, Chengru Song, et al . 2025. Cauchy: A Cost- Efficient LLM Serving System through Adaptive Heterogeneous Deployment. In Proceedings of the 2025 ACM Symposium on Cloud Computing. 881–893

    2025

  48. [48]

    Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody H Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E Gonzalez, et al . 2024. Sglang: Efficient execution of structured language model programs.Advances in neural information processing systems37 (2024), 62557–62583

    2024

  49. [49]

    Zilin Zhu, Chengxing Xie, Xin Lv, and slime Contributors. 2025. slime: An LLM post-training framework for RL Scaling. https://github.com/THUDM/slime. GitHub repository. Corresponding author: Xin Lv. 13

    2025