Pith. sign in

REVIEW 4 major objections 4 minor 37 references

Batched robot policy serving should be a closed-loop scheduling problem, and a lookahead scheduler recovers up to 18% throughput on heterogeneous fleets.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 00:40 UTC pith:33C2WE4G

load-bearing objection Solid new formulation, honest system work, but the headline real-world gain is likely a naive-async artifact the authors themselves flag. the 4 major comments →

arxiv 2608.00337 v1 pith:33C2WE4G submitted 2026-07-31 cs.RO

Action Chunk Scheduling for Batched Robot Policy Serving

Rohan Bansal , David He , Nadun Ranawaka Arachchige , Zhenyang Chen , Soobum Kim , Kexin Rong , Danfei Xu This is my paper
classification cs.RO
keywords robot policy servingaction chunkingcloud inferencebatchingschedulingclosed-loop controlheterogeneous fleetsstarvation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper's claim is that when one GPU serves many robots, batching decisions should be made by simulating how each batch changes each robot's future action buffer, not by generic heuristics. The authors formulate batched policy serving as a Markov Decision Process whose state mirrors each robot's action queue, executed index, and network delay, and whose reward is total robot time executed. They propose Lookahead, which rolls candidate batches forward and picks the one with the best reward per unit inference time. In simulation and on ten real robots, simple round-robin and deadline scheduling match Lookahead when fleets are homogeneous, but under heterogeneous reactivity demands Lookahead raises real-world system throughput by up to 18% and exposes a tunable weight for trading off fast and slow robots. The paper matters because it reframes multi-robot policy deployment as a control-aware systems problem rather than a pure throughput problem.

Core claim

The paper's central discovery is that batching decisions in a shared GPU policy server alter robot behavior in closed loop: a late chunk starves a robot, and the harm depends on the robot's task. The authors build Armory, which maintains a server-side mirror of each robot's action queue, latest executed index, in-flight chunks, and communication delay, and cast scheduling as an MDP whose reward is total executed robot time. Their Lookahead scheduler simulates candidate batches forward and dispatches the first batch of the highest-scoring schedule, normalizing reward by inference time. Evaluations on a simulation benchmark and a ten-robot real fleet show that when all robots consume chunks at

What carries the argument

The central object is the server-side execution mirror: for each robot, the action index of the latest executed step, the latest step received, and the queue of generated chunks with arrival times—where an action chunk is a sequence of future actions produced by one policy query. Scheduling is an MDP over these states; a batch transition advances robots' indices only when a covering chunk has arrived, and the per-epoch reward is total executed robot time, optionally weighted per task. Lookahead is a one-step receding-horizon algorithm (the paper finds deeper lookahead does not improve results) that scores candidate batches by predicted reward divided by inference time and dispatches the best

Load-bearing premise

The real-world 18% throughput gain is assumed to come from Lookahead's better batch selection, but the paper's own appendix shows Lookahead also avoids mid-chunk starts; if that execution smoothness is the true cause, the gain does not validate the scheduling mechanism.

What would settle it

Take the heterogeneous real-world setting and force all schedulers to execute only from chunk start—or use a chunking scheme without the index-0 bias—then compare throughput; if the Lookahead advantage disappears or shrinks to noise, the scheduling policy was not the driver.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Operators can choose a weight per robot class and move along a throughput/starvation tradeoff instead of accepting whatever a fixed heuristic delivers.
  • In homogeneous fleets the paper's experiments indicate round-robin or earliest-deadline scheduling is sufficient, so the extra machinery is only needed when reactivity demands differ.
  • Choosing small, efficient batches matters more than filling the batch: Lookahead keeps throughput flat as the batch-size cap grows while fixed-batch heuristics starve robots.
  • Because Armory is model-agnostic, its scheduling gains compose with any per-model inference speedup.
  • Network jitter and median delays up to about 50 ms degrade all schedulers similarly; scheduling policy cannot compensate for raw transport delay, but the system remains practical.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's own appendix flags that Lookahead tends to start chunks at executed index 0, avoiding the jerky mid-chunk starts that naive asynchronous execution produces; if that execution-pattern effect rather than batch selection drives the measured gains, the advantage may vanish under real-time chunking strategies that eliminate the bias.
  • A natural next step is closed-loop weight adaptation: use task-phase or progress signals to raise a robot's weight during dynamic segments and lower it when quasi-static, which the MDP formulation already supports.
  • The same mirror-based formulation could extend to multi-GPU or disaggregated serving by treating prefill/decode or model shards as scheduling resources, though the paper only assumes a single GPU.
  • Because starvation cost is task-dependent, scheduler weights could be learned from policy-success statistics instead of being set by hand, making the tradeoff data-driven.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. This paper introduces Armory, a serving system for batched inference of robot policies (e.g., VLA models) on a single remote GPU shared by multiple robots. The authors formulate action-chunk scheduling as an MDP over server-side robot state (action indices, queues, latency estimates) and propose Lookahead, a one-step receding-horizon scheduler that scores candidate batches by predicted total executed robot time normalized by inference latency. They evaluate Lookahead against Round Robin and EDF in simulation (LIBERO) and on ten real AgileX PiPER arms with a π0.5 policy. The main empirical claims are that simple schedulers suffice for homogeneous fleets, while Lookahead with tuned per-robot weights improves fast-robot service and, in the one-fast real-world configuration, increases system throughput by up to 18% over baselines.

Significance. If the empirical claims hold, the paper opens a useful new problem area: multi-robot closed-loop policy serving, where inference latency changes robot behavior rather than being a pure SLO. The MDP formulation and the software-mirror abstraction are coherent, and the paper provides detailed pseudocode, extensive ablations (batch-size cap, lookahead depth, network delay/jitter), and a real-robot testbed with two task types. The claimed 18% throughput gain in heterogeneous settings is the paper's headline contribution and would be of interest to the robotics-systems community. However, the real-world evidence currently does not establish the causal mechanism behind that gain, and the simulation results do not reproduce it, so the significance is contingent on additional experiments.

major comments (4)
  1. [§4.2 / Appendix A.8, Table 5, Figure 10] The headline 18% throughput gain is not attributable to the MDP/lookahead mechanism because the paper itself identifies an uncontrolled confound: Lookahead dispatches a large fraction of chunks starting at index 0, while RR and EDF frequently start mid-chunk; with naive async execution, mid-chunk starts cause jerky/discontinuous motions that reduce task success. Figure 10 shows a strong first-executed-index bias, and the text states that using RTC or VLASH 'may lead to different results.' This means the measured gain could be an execution-pattern side effect rather than the result of smarter starvation-aware batch selection. The authors should either control the first-executed-index distribution across schedulers, repeat with RTC/VLASH, or otherwise provide evidence that the scheduling decision drives the throughput difference.
  2. [§4.1 / Table 5 / A.8] The real-world throughput result rests on three one-minute rollouts per condition, with no error bars, confidence intervals, or significance tests. The appendix explicitly notes that 'metrics may be noisy due to small sample size' and that real-world evaluations are noisy. For the key one-fast comparison (LA@5 total 98.67 vs RR 83.33), the reader cannot tell whether the 18% difference is signal or seed variation. The paper should report per-seed results, variance, and at least a simple statistical comparison, or temper the throughput claim accordingly.
  3. [§4.2 vs Appendix A.7, Table 4] The simulation results contradict the claimed real-world transfer. Table 4 and Figure 9 show that Lookahead does not improve aggregate system throughput in simulation; Appendix A.7 explicitly states that 'the core contribution of Lookahead is not in raising aggregate throughput, but in reallocating it across tiers.' Yet Section 4.2 says 'Scheduling gains transfer from simulation to real hardware' and the abstract claims a throughput improvement. This inconsistency undercuts the claim that the lookahead mechanism itself improves total throughput. The authors should either present simulation evidence of aggregate gains or revise the transfer claim to reflect the real evidence.
  4. [§4.1 / Appendix A.4] The choice of batch-size cap may favor Lookahead independently of its scheduling logic. In the heterogeneous real-world experiments b=5, while Appendix A.4 shows that Lookahead chooses smaller batches on average (mean 3.3) whereas EDF and RR gravitate toward the cap. Since the paper notes that over-batching causes starvation, the advantage could come from Lookahead's dynamic batch-size selection rather than from its forward simulation of execution state. The claim that b=5 'favors RR and EDF' is not backed by the reported real-world sweep. Please show the batch-size tuning for the real-world baselines or run an ablation that restricts Lookahead to the same batch-size choices as the baselines.
minor comments (4)
  1. [§2 / §A.8] The term 'naive async' is used in the appendix and in the confound discussion but is not formally defined in the main text. Please define it early (e.g., in Section 2 or 3) so the reader understands the execution model that the confounding discussion relies on.
  2. [§3.2, Eq. (13)] The score in Eq. (13) normalizes by d_infer only, not by the full epoch time including communication delays. This is reasonable for a heuristic, but the choice should be justified, especially since Section 3.1's transition includes d_obs and d_action.
  3. [Figure 4 caption] The caption says 'legos per minute' while the text uses 'successes per minute' and 'Lego pieces per minute.' Use consistent units and clarify what 'success' means for the real-world tasks.
  4. [References] Reference [12] is a personal blog post. For a systems claim about execution index continuity, a peer-reviewed or archival source would be more appropriate if one exists.

Circularity Check

0 steps flagged

No significant circularity: the MDP/Lookahead derivation is self-contained; the Appendix A.8 confound is a validity threat, not a circular reduction.

full rationale

I walked the claimed derivation chain: the serving problem is formulated as an MDP (Eqs. 5-11) with reward R(s_k, B_k) = (1/f_c) sum_j w_j Delta_i_j(s_k, B_k), i.e., executed robot time. Lookahead then selects batches by forward-simulating that same transition model and maximizing Score(S) = sum R(s_l, B_l)/d_infer(|B_l|) (Eq. 13). No parameter of this model is fitted to the headline throughput numbers: d_infer(b) is profiled on startup (Section 3.3), d_obs/d_action are continuously estimated, and w_j is an operator-set knob swept as w=1,3,5 (Section 4.1). Thus the reported 'up to 18%' throughput improvement is an externally measured outcome, not a quantity reconstructed from the model's own fitted values. The only overlapping-author citation, [24] (SAIL), appears in Related Work as one of several orthogonal latency-tolerance techniques and is not load-bearing for Armory's central claims. The paper itself flags a real confound in Appendix A.8: with naive async, Lookahead's bias toward first-executed index 0 may reduce jerky motions, and 'Predicting chunks with more sophisticated asynchronous strategies such as RTC [8] or VLASH [9] may lead to different results.' I weigh this as an external-validity/causal-attribution concern, not a circularity: it does not reduce the throughput claim to the scheduler's inputs by definition. Small sample size (three 1-minute rollouts, Table 5) is an evidence-strength issue, not circularity.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The central claim rests on a domain model (action-chunk execution, starvation, task-dependent sensitivity), a chosen reward, and several experimental knobs (batch cap, weights, horizons, lookahead depth). None of the numbers are fitted to the headline throughput result, so circularity is low. The empirical generality of the gain is load-bearing on the mirror's accuracy and on the un-confounded scheduling effect, both of which are only partially validated.

free parameters (5)
  • batch-size cap b = b=3 in simulation; b=3 for real all-fast; b=5 for real heterogeneous
    Chosen by sweeping to 'give the naive methods the best shot' (Section 4.1, Appendix A.4). The main heterogeneous real-world comparison uses b=5, and headline gains could depend on this choice.
  • fast-robot reward weight w_j = w = 1, 3, 5
    User-selected weighting that controls the fast/slow throughput tradeoff; LA@5 is the setting behind the 18% claim. It is a control knob, not fitted, but it defines the reported operating point.
  • Lookahead search depth L = L=1
    Chosen after ablation (Appendix A.5) because longer lookahead slightly increased starvation in their implementation. The framework is only instantiated at one step.
  • execution horizons and control frequencies = sim Hmax=6/10, fc=20, Hmin=1; real Hmax=10/20, fc=30, Hmin=5
    Hand-selected to define fast and slow robot classes. The heterogeneity result depends on these choices being representative of real robot tasks.
  • score window and candidate-batch pruning = 1-second window; unspecified pruning heuristic
    Appendix A.1 states score_schedule uses a 1-second window and candidate_batches(s) heuristically prunes batches, but the pruning details are unstated, affecting which schedules are considered.
axioms (5)
  • domain assumption Robot execution follows action-chunk semantics: action indexes strictly increment, chunks cover [i_start, i_start+H_max), and execution pauses during starvation.
    Used in the MDP transition (Eq. 8) and throughout the software mirror. This is grounded in action-chunking literature [7,8,12] but is not re-derived in this paper.
  • domain assumption Task performance degrades monotonically with stale horizons and starvation, with dynamic tasks affected more than static tasks.
    Hypothesized in Section 2 and empirically shown in Figure 3, but the entire heterogeneity motivation and the fast/slow distinction depend on this task-dependence.
  • domain assumption Inference latency d_infer(b) is known/profiled and stable, and network delays can be estimated accurately enough for the mirror.
    Armory profiles latencies on startup and continuously estimates delays (Section 3.3). Scheduling decisions rely on the mirror's accuracy, which is asserted from the push-based architecture.
  • ad hoc to paper The serving objective is to maximize total executed robot time (weighted), and any non-empty batch can be scheduled each epoch.
    Reward Eq. (10) is a design choice, not a consequence of first principles. Maximizing executed robot time is a reasonable but selected objective.
  • ad hoc to paper One-step lookahead (L=1) is sufficient to realize the claimed scheduling benefit.
    Appendix A.5 shows longer lookahead does not help in this implementation, despite the MDP being defined for multi-step planning. The theoretical 'lookahead' claim is therefore only empirically instantiated at one step.

pith-pipeline@v1.3.0-alltime-deepseek · 16355 in / 12741 out tokens · 111249 ms · 2026-08-04T00:40:43.172319+00:00 · methodology

0 comments
read the original abstract

Deploying robot foundation models at scale is the next step towards realizing the potential of general-purpose robots. However, Vision-Language-Action (VLA) and other foundation models are computationally demanding, and on-device compute is constrained by power and space. In this paper, we introduce the problem of serving a robot policy to multiple robots from a remote GPU and formulate it as a scheduling problem. We build Armory, a serving system validated on fleets of both simulated and real robots. Our experiments show that naive scheduling heuristics perform well when all robots are the same, but fall short when robots consume action chunks at different rates, uncovering a mismatch between conventional batching methods and the closed-loop requirements of robot policy execution. To address this, we propose a scheduling algorithm that accounts for this heterogeneity and improves overall system throughput by up to $18\%$ in real-world experiments. Additional details are available at https://gatech-rl2.github.io/actionchunkscheduling.

Figures

Figures reproduced from arXiv: 2608.00337 by Danfei Xu, David He, Kexin Rong, Nadun Ranawaka Arachchige, Rohan Bansal, Soobum Kim, Zhenyang Chen.

Figure 1
Figure 1. Figure 1: Armory is an end-to-end serving system for deploying robot foundation model policies from the cloud [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Execution Timeline. A robot with Hmax = 8 consumes actions by incrementing the action index, pausing when it is starved. When starvation occurs while a chunk is still in inference/network, the chunk’s execution will extend more than Hmax steps after the observation was sent. Delay. In the real-world, there exists some delay d from the time the robot captures observation ot to the time it executes an action… view at source ↗
Figure 3
Figure 3. Figure 3: Dynamic task through￾put is more sensitive to stale ex￾ecution horizons and starvation than that of quasi-static tasks. In this section, we describe the evaluation of our suite of schedul￾ing algorithms in real-world and simulation experiments under dif￾ferent levels of robot heterogeneity. We find that under a homoge￾neous workload, simpler algorithms may suffice, while under het￾erogeneous workloads, the… view at source ↗
Figure 4
Figure 4. Figure 4: We compare system throughput in legos-per-minute across the three heterogeneous scenarios in a [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Real-world task setup. Top is the static brick-into-bin sorting task, and bottom is the dynamic brick-into-mug turntable task. Real World. We deploy 10 AgileX PiPER arms with top and wrist cameras (Intel RealSense D435i), served by a π0.5 checkpoint finetuned at H = 20 and fc = 30. Both classes use Hmin = 5 to prevent excessively mul￾timodal motions; fast robots use Hmax = 10 and slow robots use Hmax = 20.… view at source ↗
Figure 6
Figure 6. Figure 6: LIBERO-10 simulation results. For one-fast, LA@5 is able to maintain a consistently higher through [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: How does the max batch size affect scheduler performance? We ablate this on the all-fast scenario. [PITH_FULL_IMAGE:figures/full_fig_p015_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Network ablation study showing how network delay affects cloud serving for [PITH_FULL_IMAGE:figures/full_fig_p017_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: System throughput (successes/min) and average starvation as the cluster scales from [PITH_FULL_IMAGE:figures/full_fig_p018_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Lookahead schedulers bias towards scheduling chunks so that they are executed starting at index 0. [PITH_FULL_IMAGE:figures/full_fig_p020_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

37 extracted references · 17 linked inside Pith

  1. [1]

    Zitkovich, T

    B. Zitkovich, T. Yu, S. Xu, P. Xu, T. Xiao, F. Xia, J. Wu, P. Wohlhart, S. Welker, A. Wahid, et al. Rt-2: Vision-language-action models transfer web knowledge to robotic control. In Conference on Robot Learning, pages 2165–2183. PMLR, 2023

    2023

  2. [2]

    M. J. Kim, K. Pertsch, S. Karamcheti, T. Xiao, A. Balakrishna, S. Nair, R. Rafailov, E. Foster, G. Lam, P. Sanketi, et al. Openvla: An open-source vision-language-action model.arXiv preprint arXiv:2406.09246, 2024

    Pith/arXiv arXiv 2024

  3. [3]

    Black, N

    K. Black, N. Brown, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, L. Groom, K. Hausman, B. Ichter, et al. pi0: A vision-language-action flow model for general robot control.arXiv preprint arXiv:2410.24164, 2024

    Pith/arXiv arXiv 2024

  4. [4]

    Intelligence, K

    P. Intelligence, K. Black, N. Brown, J. Darpinian, K. Dhabalia, D. Driess, A. Esmail, M. Equi, C. Finn, N. Fusai, et al. pi0.5: A vision-language-action model with open-world generalization.arXiv preprint arXiv:2504.16054, 2025

    Pith/arXiv arXiv 2025

  5. [5]

    Bjorck, F

    J. Bjorck, F. Casta ˜neda, N. Cherniadev, X. Da, R. Ding, L. Fan, Y . Fang, D. Fox, F. Hu, S. Huang, et al. Gr00t n1: An open foundation model for generalist humanoid robots.arXiv preprint arXiv:2503.14734, 2025

    Pith/arXiv arXiv 2025

  6. [6]

    Jiang, J

    W. Jiang, J. Clemons, K. Sankaralingam, and C. Kozyrakis. How fast can i run my vla? demystifying vla inference performance with vla-perf.arXiv preprint arXiv:2602.18397, 2026

    arXiv 2026

  7. [7]

    T. Z. Zhao, V . Kumar, S. Levine, and C. Finn. Learning fine-grained bimanual manipulation with low-cost hardware.arXiv preprint arXiv:2304.13705, 2023

    Pith/arXiv arXiv 2023

  8. [8]

    Black, M

    K. Black, M. Y . Galliker, and S. Levine. Real-time execution of action chunking flow policies. arXiv preprint arXiv:2506.07339, 2025

    Pith/arXiv arXiv 2025

  9. [9]

    J. Tang, Y . Sun, Y . Zhao, S. Yang, Y . Lin, Z. Zhang, J. Hou, Y . Lu, Z. Liu, and S. Han. Vlash: Real-time vlas via future-state-aware asynchronous inference.arXiv preprint arXiv:2512.01031, 2025

    Pith/arXiv arXiv 2025

  10. [10]

    Black, A

    K. Black, A. Z. Ren, M. Equi, and S. Levine. Training-time action conditioning for efficient real-time chunking.arXiv preprint arXiv:2512.05964, 2025

    arXiv 2025

  11. [11]

    Agrawal, A

    A. Agrawal, A. Panwar, J. Mohan, N. Kwatra, B. S. Gulavani, and R. Ramjee. Sarathi: Efficient llm inference by piggybacking decodes with chunked prefills.arXiv preprint arXiv:2308.16369, 2023

    Pith/arXiv arXiv 2023

  12. [12]

    J. Vial. Distributed real-time chunking, Mar 2026. URLhttps://jackvial.com/posts/ distributed-real-time-chunking.html

    2026

  13. [13]

    C. Chi, Z. Xu, S. Feng, E. Cousineau, Y . Du, B. Burchfiel, R. Tedrake, and S. Song. Diffusion policy: Visuomotor policy learning via action diffusion.The International Journal of Robotics Research, 44(10-11):1684–1704, 2025

    2025

  14. [14]

    T. T. Zhang, D. Pfrommer, C. Pan, N. Matni, and M. Simchowitz. Action chunking and ex- ploratory data collection yield exponential improvements in behavior cloning for continuous control.arXiv preprint arXiv:2507.09061, 2025

    arXiv 2025

  15. [15]

    B. Liu, Y . Zhu, C. Gao, Y . Feng, Q. Liu, Y . Zhu, and P. Stone. Libero: Benchmarking knowl- edge transfer for lifelong robot learning.Advances in Neural Information Processing Systems, 36:44776–44791, 2023. 10

    2023

  16. [16]

    M. J. Kim, Y . Gao, T.-Y . Lin, Y .-C. Lin, Y . Ge, G. Lam, P. Liang, S. Song, M.-Y . Liu, C. Finn, et al. Cosmos policy: Fine-tuning video models for visuomotor control and planning.arXiv preprint arXiv:2601.16163, 2026

    Pith/arXiv arXiv 2026

  17. [17]

    O’Neill, A

    A. O’Neill, A. Rehman, A. Maddukuri, A. Gupta, A. Padalkar, A. Lee, A. Pooley, A. Gupta, A. Mandlekar, A. Jain, et al. Open x-embodiment: Robotic learning datasets and rt-x models: Open x-embodiment collaboration 0. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 6892–6903. IEEE, 2024

    2024

  18. [18]

    Khazatsky, K

    A. Khazatsky, K. Pertsch, S. Nair, A. Balakrishna, S. Dasari, S. Karamcheti, S. Nasiriany, M. K. Srirama, L. Y . Chen, K. Ellis, et al. Droid: A large-scale in-the-wild robot manipulation dataset.arXiv preprint arXiv:2403.12945, 2024

    Pith/arXiv arXiv 2024

  19. [19]

    S. Ye, Y . Ge, K. Zheng, S. Gao, S. Yu, G. Kurian, S. Indupuru, Y . L. Tan, C. Zhu, J. Xiang, et al. World action models are zero-shot policies.arXiv preprint arXiv:2602.15922, 2026

    Pith/arXiv arXiv 2026

  20. [20]

    J. Kwok, C. Agia, R. Sinha, M. Foutter, S. Li, I. Stoica, A. Mirhoseini, and M. Pavone. Robomonkey: Scaling test-time sampling and verification for vision-language-action models. arXiv preprint arXiv:2506.17811, 2025

    Pith/arXiv arXiv 2025

  21. [21]

    Y . Ma, Y . Zhou, Y . Yang, T. Wang, and H. Fan. Running vlas at real-time speed.arXiv preprint arXiv:2510.26742, 2025

    arXiv 2025

  22. [22]

    S. H. Høeg, Y . Du, and O. Egeland. Streaming diffusion policy: Fast policy synthesis with variable noise diffusion models.arXiv preprint arXiv:2406.04806, 2024

    Pith/arXiv arXiv 2024

  23. [23]

    L. Guo, Z. Xue, Z. Xu, and H. Xu. Demospeedup: Accelerating visuomotor policies via entropy-guided demonstration acceleration. InProceedings of The 9th Conference on Robot Learning, volume 305 ofProceedings of Machine Learning Research, pages 599–609. PMLR, 2025

    2025

  24. [24]

    N. R. Arachchige, Z. Chen, W. Jung, W. C. Shin, R. Bansal, P. Barroso, Y . H. He, Y . C. Lin, B. Joffe, S. Kousik, et al. Sail: Faster-than-demonstration execution of imitation learning policies.arXiv preprint arXiv:2506.11948, 2025

    Pith/arXiv arXiv 2025

  25. [25]

    Crankshaw, X

    D. Crankshaw, X. Wang, G. Zhou, M. J. Franklin, J. E. Gonzalez, and I. Stoica. Clipper: A low-latency online prediction serving system. In14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17). USENIX Association, 2017

    2017

  26. [26]

    Crankshaw, G.-E

    D. Crankshaw, G.-E. Sela, C. Zumar, X. Mo, J. E. Gonzalez, I. Stoica, and A. Tumanov. Inferline: Ml prediction pipeline provisioning and management for tight latency objectives. arXiv preprint arXiv:1812.01776, 2018

    Pith/arXiv arXiv 2018

  27. [27]

    Gujarati, R

    A. Gujarati, R. Karimi, S. Alzayat, W. Hao, A. Kaufmann, Y . Vigfusson, and J. Mace. Serving dnns like clockwork: Performance predictability from the bottom up. In14th USENIX Sympo- sium on Operating Systems Design and Implementation (OSDI 20), pages 443–462. USENIX Association, 2020

    2020

  28. [28]

    G.-I. Yu, J. S. Jeong, G.-W. Kim, S. Kim, and B.-G. Chun. Orca: A distributed serving system for{Transformer-Based}generative models. In16th USENIX symposium on operating systems design and implementation (OSDI 22), pages 521–538, 2022

    2022

  29. [29]

    W. Kwon, Z. Li, S. Zhuang, Y . Sheng, L. Zheng, C. H. Yu, J. Gonzalez, H. Zhang, and I. Sto- ica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th symposium on operating systems principles, pages 611–626, 2023

    2023

  30. [30]

    Zheng, L

    L. Zheng, L. Yin, Z. Xie, C. L. Sun, J. Huang, C. H. Yu, S. Cao, C. Kozyrakis, I. Stoica, J. E. Gonzalez, et al. Sglang: Efficient execution of structured language model programs.Advances in neural information processing systems, 37:62557–62583, 2024. 11

    2024

  31. [31]

    Zhong, S

    Y . Zhong, S. Liu, J. Chen, J. Hu, Y . Zhu, X. Liu, X. Jin, and H. Zhang.{DistServe}: Disag- gregating prefill and decoding for goodput-optimized large language model serving. In18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24), pages 193– 210, 2024

    2024

  32. [32]

    C. L. Liu and J. W. Layland. Scheduling algorithms for multiprogramming in a hard-real-time environment.Journal of the ACM (JACM), 20(1):46–61, 1973

    1973

  33. [33]

    Tumanov, T

    A. Tumanov, T. Zhu, J. W. Park, M. A. Kozuch, M. Harchol-Balter, and G. R. Ganger. TetriSched: Global rescheduling with adaptive plan-ahead in dynamic heterogeneous clus- ters. InProceedings of the 11th European Conference on Computer Systems. Association for Computing Machinery, 2016

    2016

  34. [34]

    Narayanan, R

    D. Narayanan, R. Rao, S. Kandula, A. Seshadri, S. Roberts, P. Chaudhary, J. Gu, J. Gonzalez, A. Harlap, A. Krishnamurthy, et al. Heterogeneity-aware cluster scheduling policies for deep learning workloads. In14th USENIX Symposium on Operating Systems Design and Implemen- tation (OSDI 20), pages 481–498. USENIX Association, 2020

    2020

  35. [35]

    Y . Dai, G. Ananthanarayanan, L. Cox, X. Foukas, B. Radunovic, and R. Netravali. Kairos: A scalable serving system for physical ai.arXiv preprint arXiv:2605.11381, 2026

    Pith/arXiv arXiv 2026

  36. [36]

    Jiang, J

    W. Jiang, J. Clemons, R. O’Flaherty, H. Hadfield, A. Degirmenci, S. Song, Y . Narang, and C. Kozyrakis. Rosa: A robotics foundation model serving system for robot factories.arXiv preprint arXiv:2607.01088, 2026

    Pith/arXiv arXiv 2026

  37. [37]

    Patel, E

    P. Patel, E. Choukse, C. Zhang, A. Shah, ´I. Goiri, S. Maleki, and R. Bianchini. Splitwise: Efficient generative llm inference using phase splitting. In2024 ACM/IEEE 51st Annual Inter- national Symposium on Computer Architecture (ISCA), pages 118–132. IEEE, 2024. 12 A Appendix Contents A.1 Scheduler Pseudocode . . . . . . . . . . . . . . . . . . . . . . ....

    2024