Pith. sign in

REVIEW 4 major objections 4 minor 109 references

MQFQ-Sticky: Fair Queueing For Serverless GPU Functions

T0 review · 4 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MQFQ-Sticky schedules GPU functions with I/O-style fair queueing and reports 2x-20x latency reductions.

desk verdict MQFQ-Sticky is a solid systems contribution that adapts fair queueing to GPU function dispatch; the main caveat is that the 'black-box' claim is narrower than advertised because the shim only intercepts cuMemAlloc. read the letter →

arxiv 2507.08954 v1 pith:KPI56AGZ submitted 2025-07-11 cs.DC cs.SYeess.SY

Alexander Fuerst , Siddharth Anil , Vishakha Dixit , Purushottam (Puru) Kulkarni , Prateek Sharma This is my paper
classification cs.DCcs.SYeess.SY
keywords MQFQ-StickyserverlesscomputingGPUschedulingfairqueueinganticipatoryUnifiedVirtualMemorymanagementwarmcontainerpool
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that GPU-accelerated serverless functions can be scheduled efficiently without modifying user code, by transplanting fair-queueing ideas from disk I/O scheduling into the FaaS control plane. It argues that GPU function dispatch has the same locality-versus-fairness tension as I/O scheduling, and that a locality-enhanced variant of multi-queue fair queuing, paired with proactive Unified Virtual Memory prefetch and swap, resolves it. The authors claim this reduces average function latency by 2x to 20x compared with existing GPU and CPU queueing policies, and by more than 300x compared with unoptimized GPU containers. If correct, serverless platforms could offer near-specialized-inference latency and utilization while still running arbitrary black-box functions.

What carries the argument

The central object is MQFQ (Multi-Queue Fair Queuing) transplanted from I/O scheduling, adapted to GPU function dispatch. Each function gets a queue with a virtual time (VT) equal to accumulated GPU service time; a dispatcher picks among queues whose VT is within T of the global minimum VT, sorts by backlog length then in-flight count, and the device-parallelism parameter D controls how many invocations run concurrently. Two GPU-specific mechanisms carry the performance argument: anticipatory scheduling keeps empty queues alive for a TTL proportional to alpha times the function's inter-arrival time, and an integrated memory manager uses an LD_PRELOAD shim that rewrites cuMemAlloc to cuMemAllocManaged, prefetching active functions' memory to the GPU and asynchronously swapping throttled or inactive memory back to host via LRU. The fairness guarantee, inherited from MQFQ, bounds the difference in normalized GPU service time across any two backlogged functions.

What would settle it

Run a workload of CUDA functions that link the CUDA runtime statically or allocate memory through low-level interfaces the shim does not intercept, and check whether they run correctly and keep the reported 2x-20x gains; one such function failing or showing the kind of 30% slowdown measured for Srad would break the black-box claim.

Watch

Extended reading notes

Core claim

The paper's central claim is that MQFQ-Sticky, an integrated GPU function scheduler and memory manager, balances the tradeoffs between locality, fairness, and latency for black-box containerized functions. Viewing the GPU as a multi-queue I/O device, it keeps one dispatch queue per function, tracks virtual service time per queue, throttles queues that exceed a global minimum by a tunable overrun T, and dispatches candidate queues by longest queue then fewest in-flight invocations to create batching and stickiness. An anticipatory keep-alive TTL keeps empty queues alive in proportion to their inter-arrival time, and a warm container pool with CUDA UVM prefetch/swap moves function memory onto and off the device ahead of execution. The authors show empirically that service time across functions tracks the MQFQ fairness bound, and that the policy reduces latency versus FCFS, continuous batching, and Paella-style fair SJF on Zipfian and Azure-sampled workloads.

Load-bearing premise

The design assumes that any black-box CUDA function can be run through a shim that swaps its graphics-memory allocations for unified virtual memory with negligible overhead and identical results; functions that bypass that swap are not covered.

Editorial extensions

If this is right

  • If MQFQ-Sticky works as claimed, GPU FaaS platforms can keep a warm container pool inside a few tens of GB of VRAM by swapping to host memory, cutting cold-start latency by orders of magnitude for black-box functions.
  • Fairness is preserved: the difference in GPU service time received by any two backlogged functions stays within the MQFQ bound, so popular functions cannot starve rare ones.
  • The scheduler composes with hardware multiplexing layers: with MPS it reduces latency by up to 80%, with MIG it exposes the slowdown small slices cause, and with multiple GPUs it extends locality-aware dispatch to cut queuing by up to 4x.
  • The parameter space of overrun T, parallelism D, keep-alive TTL, and pool size has a wide stable region, so operators can tune for locality versus fairness without brittle configurations.
  • All comparison policies in the evaluation use the same memory-management and shim optimizations, isolating queueing policy as the differentiator.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper, the GPU-as-multi-queue-I/O-device analogy suggests cluster-level load balancers could use per-server virtual-time state instead of consistent hashing to preserve locality across servers; MQFQ-Sticky currently treats load balancing as separate.
  • A testable extension is applying the same anticipatory TTL and queue-overrun parameters to other accelerators with scarce on-device memory and expensive context initialization, such as TPU or NPU slices with unified addressing; the paper only evaluates NVIDIA GPUs.
  • The measured 30% Srad shim overhead implies a production deployment would need a non-UVM fallback path for UVM-sensitive kernels, and the paper does not quantify how such fallbacks would affect the fairness bound.
  • Because MQFQ-Sticky's dispatch heuristic selects deterministically among queues that are MQFQ-legal, a tighter fairness bound than the inherited one may be derivable; the paper notes this possibility and leaves it open.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper presents MQFQ-Sticky, a GPU function scheduler for serverless FaaS that adapts Multi-Queue Fair Queuing and anticipatory scheduling to the GPU setting, and integrates it with UVM-based memory prefetching and swapping inside a container pool. The system targets black-box containerized CUDA functions, claims to balance locality, fairness, and latency, and reports latency reductions of 2x--20x over existing queueing policies and over 300x over unoptimized GPU containers. The evaluation uses two GPU platforms, a mix of machine-learning, video, and HPC functions, open-loop Zipfian and Azure-derived workloads, and a sensitivity study of the main parameters. The central design idea is plausible and the empirical scope is broad, but several issues in the pseudocode, the fairness argument, and the claimed black-box generality need to be addressed.

Significance. If the central claims hold, this is a useful contribution to serverless GPU systems: it provides a principled I/O-scheduling-inspired approach to GPU function scheduling, integrates memory management with queue state, and demonstrates robustness across parameter choices and hardware features. The work is distinctive in aiming for black-box function support rather than application-specific optimizations, and it explicitly measures fairness, locality, and latency tradeoffs. The empirical evaluation is broad, including two GPU generations, MPS/MIG, multi-GPU settings, and multiple workload classes. However, the significance is tempered by the fact that the fairness bound is inherited from prior work rather than derived, the black-box claim is only validated for cuMemAlloc-based functions, and the experimental reporting lacks confidence intervals and an artifact release.

major comments (4)
  1. [Section 5.1 / Section 3.1] The black-box generality claim is broader than what the implementation supports. Section 5.1 states that the LD_PRELOAD shim intercepts only cuMemAlloc and rewrites it to cuMemAllocManaged, and records metadata for memory management. Functions that use the CUDA VMM API (cuMemCreate/cuMemMap), CUDA graphs with memory pools, custom allocators layered above cuMemAlloc, or statically linked CUDA runtimes will bypass the shim entirely; for those functions, the scheduler cannot track or prefetch/swap GPU memory, so the UVM-based oversubscription and proactive memory management do not apply. The paper's abstract and Section 3.1 claim black-box operation without modifying function code, and the latency and cold-start improvements in the evaluation depend directly on the memory management in Section 4.3. This is a load-bearing limitation: the 2x--20x latency claim is demonstrated only for functions whose allocations go through cuMemAlloc. The paper should either state this scope explicitly in the abstract and Section 3.1, or extend the shim to cover the VMM API and CUDA graph memory pools, and discuss the expected impact on the reported results. The 30% overhead reported for Srad in Figure 3 also shows that the 'negligible overhead' characterization is not universally true even within the supported class.
  2. [Algorithm 1] There is a direct inconsistency between the candidate-selection predicate and the state-update rule. Line 6 selects active queues with queue.VT < Global_VT + T, which is the eligibility condition described in the text. Line 22, however, sets a queue to Throttled when queue.VT - Global_VT < T, which is exactly the same condition under a sign-inverted formulation; as written, the queues that line 6 would select are marked throttled, while queues with queue.VT - Global_VT >= T are marked active even though line 6 would not dispatch them. This appears to be a sign error in the pseudocode, but since Algorithm 1 is the formal specification of the proposed scheduler, it must be corrected and the corrected version should be checked against the prose in Section 4.2, which itself contains the confusing statement 'If queue.VT + T >= Global_VT, then the queue is throttled'.
  3. [Section 4.2] The fairness bound is claimed to follow from the MQFQ theorem [40], but the scheduler advances virtual time using the historical average execution time tau_k rather than the actual service time of the dispatch. Section 4.2 says 'we track the historical average execution time tau_k ... and when an item is dispatched, increment its queue's VT by tau_k.' Equation (1), however, bounds the difference in actual GPU service times S_i and S_j. If the actual execution time of an invocation deviates from tau_k, the virtual-time accounting no longer reflects service actually rendered, and the MQFQ fairness bound is not automatically inherited. The paper needs to state the assumptions under which Eq. (1) holds, or restate the bound in terms of the estimated virtual time; otherwise, the 'we retain the fairness properties of MQFQ' claim in Section 4.2 is not justified. The empirical check in Figure 5b is suggestive but does not substitute for a correct theoretical statement.
  4. [Section 6] The empirical central claims would be materially strengthened by reporting variability and providing the artifact. Most figures show averages over 5 runs without confidence intervals or per-run distributions, so it is difficult to assess whether the reported 2x--5x improvements over FCFS or the differences between MQFQ-Sticky and Batch are statistically meaningful for the selected traces. Additionally, no artifact or raw data is provided, and the Paella and Batch baselines are reimplementations rather than the original systems. Given that the paper claims a new scheduler with large latency gains and a fairness guarantee, the lack of an artifact makes independent verification hard and should be addressed, at minimum by releasing the scheduler, the shim, and the workload traces, and by including confidence intervals or run-level data in the main latency comparisons.
minor comments (4)
  1. [Section 5] The text says that line 9 of Algorithm 1 implements 'sticky' load balancing among GPUs by avoiding moving functions across GPUs, but line 9 only sorts candidates by in-flight count; no GPU-affinity or sticky-binding mechanism is visible in Algorithm 1. Please clarify how the dispatcher maintains per-GPU locality.
  2. [Figures 6 and 7] The captions of Figures 6 and 7 both read 'Latency, fairness, and utilization for a medium-intensity FaaS workload', but the figures mainly show latency and utilization; no fairness measure is presented in either figure. Please make the captions reflect the content.
  3. [Section 6.2] The comparison to 'FCFS Naive' with an average latency near 3,000 seconds is described but not shown in any figure; since this is the basis for the '300x' claim, please add a table or a note describing how this number was obtained.
  4. [Section 7] There are typos in the related work section: 'stravation' should be 'starvation', and 'investigtes' should be 'investigates'.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: fairness inherits from external MQFQ theorem; latency gains are measured against baselines; only minor non-load-bearing self-citations appear.

full rationale

The paper's central claims do not reduce to their inputs. The fairness bound in Section 4.2 is explicitly imported from an external theorem: 'Based on the main theorem in [40]' and 'The proof of this property hinges on the assumption that queue.VT < Global_VT + T, which we meet in line 6.' Reference [40] is Hedayati et al., not the present authors, so this is independent support; MQFQ-Sticky restricts the dispatch choices to a subset of MQFQ's allowed choices, and the bound is inherited rather than derived from fitted data. The scheduler does consume measured historical execution times ('We track the historical average execution time τ_k of each function k, and when an item is dispatched, increment its queue’s VT by τ_k'), but this is a feedback input for virtual-time accounting, not a fitted parameter renamed as a prediction. The 2x–20x latency reductions are obtained experimentally by comparing MQFQ-Sticky with FCFS, Batch, Paella, and an Iluvatar deadline policy, all using the same memory-management optimizations. The paper's self-citations ([32] Iluvatar as the implementation base, [33] for the long-tailed reuse-distance observation, [34] for locality-aware load balancing) are contextual or baseline citations; none is presented as a uniqueness theorem or as the sole justification for the central scheduling claim. The explicit Srad 30% shim overhead in Section 5.1 is a scope/generality limitation, not evidence of circularity.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claims rest on manually configured scheduling parameters (T, D, alpha, pool size), assumptions about the GPU run-to-completion model and UVM transparency, and a cited fairness theorem that is not re-proven. No new physical or conceptual entities (particles, forces, dimensions) are introduced; the 'shim' and 'container pool' are implemented software components, not postulated entities.

free parameters (4)
  • Queue over-run threshold T = default 10
    Used in candidate filter and fairness bound Eq. (1); manual knob, sensitivity shown in Fig. 8a, not fitted to data.
  • Device concurrency D = default 2, dynamic
    Max concurrent GPU dispatches; manual with utilization feedback; affects all latency results.
  • Anticipatory TTL multiplier alpha = default 2
    Sets queue TTL = alpha * IAT; manual knob; Fig. 8b.
  • Warm container pool size = default 32
    Manual; cold-hit rate vs memory tradeoff; Fig. 8c.
assumptions (5)
  • standard math MQFQ fairness theorem (Eq. 1) transfers to MQFQ-Sticky because dispatch choices are a subset of MQFQ
    Invoked in Section 4.2 'Based on the main theorem in [40]'; no proof is reproduced, and the pseudocode's throttling condition appears inconsistent.
  • domain assumption GPU functions execute run-to-completion and GPU multiplexing is much more limited than CPUs
    Section 3.1; the queueing model and D parameter depend on this.
  • domain assumption LD_PRELOAD interposition of cuMemAlloc to cuMemAllocManaged is transparent for black-box functions
    Section 5.1; Figure 3 shows mostly small overhead but Srad has 30% overhead and alternative allocation paths are not handled.
  • domain assumption CUDA UVM can safely oversubscribe GPU memory with host-backed paging
    Section 5.2; required for the warm-pool and swap design; depends on NVIDIA driver behavior.
  • domain assumption Zipfian and sampled Azure traces are representative of GPU FaaS workloads
    Section 6 setup; used for all latency and fairness conclusions.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MQFQ-Sticky: Fair Queueing For Serverless GPU Functions." pith.science (2026). https://pith.science/paper/KPI56AGZ

@misc{pith2026250708954,
  author       = {Pith},
  title        = {Pith review of: MQFQ-Sticky: Fair Queueing For Serverless GPU Functions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KPI56AGZ}},
  note         = {Machine review of arXiv:2507.08954}
}
read the original abstract

Hardware accelerators like GPUs are now ubiquitous in data centers, but are not fully supported by common cloud abstractions such as Functions as a Service (FaaS). Many popular and emerging FaaS applications such as machine learning and scientific computing can benefit from GPU acceleration. However, FaaS frameworks (such as OpenWhisk) are not capable of providing this acceleration because of the impedance mismatch between GPUs and the FaaS programming model, which requires virtualization and sandboxing of each function. The challenges are amplified due to the highly dynamic and heterogeneous FaaS workloads. This paper presents the design and implementation of a FaaS system for providing GPU acceleration in a black-box manner (without modifying function code). Running small functions in containerized sandboxes is challenging due to limited GPU concurrency and high cold-start overheads, resulting in heavy queueing of function invocations. We show how principles from I/O scheduling, such as fair queuing and anticipatory scheduling, can be translated to function scheduling on GPUs. We develop MQFQ-Sticky, an integrated fair queueing and GPU memory management approach, which balances the tradeoffs between locality, fairness, and latency. Empirical evaluation on a range of workloads shows that it reduces function latency by 2x to 20x compared to existing GPU and CPU queueing policies.

Figures

Figures reproduced from arXiv: 2507.08954 by the authors.

Figure 1
Figure 1. Timeline of cold-starts of CPU (top) and GPU (bot [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Scheduling GPU functions from individual queues [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 4
Figure 4. Active memory management (Prefetch+Swap) im￾proves execution latency. We use CUDA’s Unified Virtual Memory (UVM) to oversubscribe device memory. UVM uses a unified host-device memory address space, with memory pointers for applications being valid in both contexts. The CUDA driver moves and ensures coherency of UVM memory between the host and device as use and pressure demands, mimicking disk-based swap space found … view at source ↗
Figures from the paper (4 more)
Figure 5
Figure 5. Figure 5: MQFQ-Sticky provides fair GPU access to functions, and also significantly reduces end-to-end latency. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Latency, fairness, and utilization for a medium-intensity FaaS workload. [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Latency, fairness, and utilization for a medium-intensity FaaS workload. [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: Queue TTL and device parallelism (D) are set based on workload and utilization. Figure 8b also shows the effect of using per-function IATs, and compares against a fixed global IAT approach, where the TTL for all functions is the same. Using the per-function IAT improve…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

109 extracted references · 60 canonical work pages

  1. [40]

    In Proceedings of the 52nd International Conference on Parallel Processing

    FaST-GShare: Enabling efficient spatio-temporal GPU sharing in serverless 12 computing for deep learning inference. In Proceedings of the 52nd International Conference on Parallel Processing . 635–644

    work page

  2. [1]

    [n. d.]. Best practices for GPU-accelerated instances. https://www.alibabacloud.com/help/en/fc/use-cases/best-practices-for-gpu- accelerated-instances/

  3. [2]

    [n. d.]. Netflix & AWS Lambda Case Study. https://aws.amazon.com/solutions/case-studies/netflix-and-aws-lambda/

  4. [3]

    [n. d.]. NVIDIA Management Library. https://developer.nvidia.com/nvidia- management-library-nvml

  5. [4]

    2015. Docker. https://www.docker.com/

    2015

  6. [5]

    Unified Memory for CUDA Beginners

    2017. Unified Memory for CUDA Beginners. https://developer.nvidia.com/blog/ unified-memory-cuda-beginners/

    2017

  7. [6]

    AWS Lambda

    2020. AWS Lambda. https://aws.amazon.com/lambda/

    2020

  8. [7]

    Nvidia container toolkit install guide

    2020. Nvidia container toolkit install guide. https://docs.nvidia.com/datacenter/ cloud-native/container-toolkit/latest/install-guide.html

    2020

Show all 109 references
  1. [8]

    Alibaba Cloud GPU Function Cold Start Overheads

    2024. Alibaba Cloud GPU Function Cold Start Overheads. https: //www.alibabacloud.com/help/en/fc/use-cases/quasi-real-time-inference- scenarios#section-rzz-zcb-w4e

    2024

  2. [9]

    Mania Abdi, Samuel Ginzburg, Xiayue Charles Lin, Jose Faleiro, Gohar Irfan Chaudhry, Inigo Goiri, Ricardo Bianchini, Daniel S Berger, and Rodrigo Fonseca

  3. [10]

    Bilge Acun, Benjamin Lee, Fiodar Kazhamiaka, Kiwan Maeng, Manoj Chakkar- avarthy, Udit Gupta, David Brooks, and Carole-Jean Wu. 2022. Carbon Ex- plorer: A Holistic Approach for Designing Carbon Aware Datacenters. http: //arxiv.org/abs/2201.10036 arXiv:2201.10036 [cs, eess]

    2022 arXiv

  4. [11]

    Alexandru Agache, Marc Brooker, Alexandra Iordache, Anthony Liguori, Rolf Neugebauer, Phil Piwonka, and Diana-Maria Popa. 2020. Firecracker: Light- weight Virtualization for Serverless Applications. In 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI ...

    2020

  5. [12]

    Georgios Alexopoulos and Dimitris Mitropoulos. 2023. nvshare: Practical GPU Sharing without Memory Size Constraints. https://github.com/grgalex/nvshare

    2023

  6. [13]

    Ahsan Ali, Riccardo Pinciroli, Feng Yan, and Evgenia Smirni. 2022. Optimizing inference serving on serverless platforms. Proceedings of the VLDB Endowment 15, 10 (2022)

    2022

  7. [14]

    Lixiang Ao, Liz Izhikevich, Geoffrey M Voelker, and George Porter. 2018. Sprocket: A serverless video processing framework. In Proceedings of the ACM Symposium on Cloud Computing. 263–274

    2018

  8. [15]

    Gabriel Aumala, Edwin Boza, Luis Ortiz-Avilés, Gustavo Totoy, and Cristina Abad. 2019. Beyond Load Balancing: Package-Aware Scheduling for Serverless Platforms. In 2019 19th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing (CCGRID). 282–291. doi:10.1109/CC...

    2019

  9. [16]

    Rachata Ausavarungnirun, Vance Miller, Joshua Landgraf, Saugata Ghose, Jayneel Gandhi, Adwait Jog, Christopher J Rossbach, and Onur Mutlu. 2018. Mask: Redesigning the gpu memory hierarchy to support multi-application concurrency. ACM SIGPLAN Notices 53, 2 (2018), 503–518

    2018

  10. [17]

    Arda Aytekin and Mikael Johansson. 2019. Harnessing the power of serverless runtimes for large-scale optimization. arXiv preprint arXiv:1901.03161 (2019)

    2019 arXiv

  11. [18]

    André Bauer, Haochen Pan, Ryan Chard, Yadu Babuji, Josh Bryan, Devesh Tiwari, Ian Foster, and Kyle Chard. 2024. The globus compute dataset: An open function- as-a-service dataset from the edge to the cloud. Future Generation Computer Systems 153 (2024), 558–574

    2024

  12. [19]

    Shiyi Cao, Yichuan Wang, Ziming Mao, Pin-Lun Hsu, Liangsheng Yin, Tian Xia, Dacheng Li, Shu Liu, Yineng Zhang, Yang Zhou, et al. 2025. Locality-aware Fair Scheduling in LLM Serving. arXiv preprint arXiv:2501.14312 (2025)

    2025 arXiv

  13. [20]

    Joao Carreira, Pedro Fonseca, Alexey Tumanov, Andrew Zhang, and Randy Katz

  14. [21]

    Ryan Chard, Yadu Babuji, Zhuozhao Li, Tyler Skluzacek, Anna Woodard, Ben Blaiszik, Ian Foster, and Kyle Chard. 2020. FuncX: A Federated Function Serving Fabric for Science. In Proceedings of the 29th International Symposium on High- Performance Parallel and Distributed Computi...

    2020

  15. [22]

    Shuai Che, Michael Boyer, Jiayuan Meng, David Tarjan, Jeremy W Sheaffer, Sang- Ha Lee, and Kevin Skadron. 2009. Rodinia: A benchmark suite for heterogeneous computing. In 2009 IEEE international symposium on workload characterization (IISWC). Ieee, 44–54

    2009

  16. [23]

    Lazar Cvetković, François Costa, Mihajlo Djokic, Michal Friedman, and Ana Klimovic. 2024. Dirigent: Lightweight Serverless Orchestration. arXiv preprint arXiv:2404.16393 (2024)

    2024 arXiv

  17. [24]

    Lazar Cvetković, Rodrigo Fonseca, and Ana Klimovic. 2023. Understanding the Neglected Cost of Serverless Cluster Management. In Proceedings of the 4th Workshop on Resource Disaggregation and Serverless (Koblenz, Germany) (WORDS ’23). Association for Computing Machinery, New Yo...

    2023

  18. [25]

    Dong Du, Qingyuan Liu, Xueqiang Jiang, Yubin Xia, Binyu Zang, and Haibo Chen. 2022. Serverless computing on heterogeneous computers. In Proceedings of the 27th ACM international conference on architectural support for programming languages and operating systems . 797–813

    2022

  19. [26]

    Dong Du, Tianyi Yu, Yubin Xia, Binyu Zang, Guanglu Yan, Chenggang Qin, Qix- uan Wu, and Haibo Chen. 2020. Catalyzer: Sub-millisecond startup for serverless computing with initialization-less booting. In Proceedings of the Twenty-Fifth International Conference on Architectural ...

    2020

  20. [27]

    José Duato, Antonio J Pena, Federico Silla, Rafael Mayo, and Enrique S Quintana- Ortí. 2010. rCUDA: Reducing the number of GPU-based accelerators in high performance clusters. In 2010 International Conference on High Performance Com- puting & Simulation. IEEE, 224–231

    2010

  21. [28]

    Ana Ebrahimi, Mostafa Ghobaei-Arani, and Hadi Saboohi. 2024. Cold Start Latency Mitigation Mechanisms in Serverless Computing: Taxonomy, Review, and Future Directions. Journal of Systems Architecture (2024), 103115

    2024

  22. [29]

    Henrique Fingler, Zhiting Zhu, Esther Yoon, Zhipeng Jia, Emmett Witchel, and Christopher J Rossbach. 2022. Dgsf: Disaggregated gpus for serverless functions. In 2022 IEEE International Parallel and Distributed Processing Symposium (IPDPS) . IEEE, 739–750

    2022

  23. [30]

    Yuqi Fu, Li Liu, Haoliang Wang, Yue Cheng, and Songqing Chen. 2022. SFS: Smart OS Scheduling for Serverless Functions. In 2022 SC22: International Conference for High Performance Computing, Networking, Storage and Analysis (SC) . IEEE Computer Society, 584–599

    2022

  24. [31]

    Yuqi Fu, Ruizhe Shi, Haoliang Wang, Songqing Chen, and Yue Cheng. 2024. Alps: An Adaptive Learning, Priority OS Scheduler for Serverless Function. In Proceedings of the 2024 USENIX Conference on Usenix Annual Technical Conference . 19–36

    2024

  25. [32]

    Alexander Fuerst, Abdul Rehman, and Prateek Sharma. 2023. Ilúvatar: A Fast Control Plane for Serverless Computing. In Proceedings of the 32nd International Symposium on High-Performance Parallel and Distributed Computing (HPDC ’23) . Association for Computing Machinery

    2023

  26. [33]

    Alexander Fuerst and Prateek Sharma. 2021. FaasCache: Keeping Serverless Computing Alive with Greedy-Dual Caching. In Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems (Virtual, USA) (ASPLOS 2021). Ass...

    2021

  27. [34]

    Alexander Fuerst and Prateek Sharma. 2022. Locality-aware Load-Balancing For Serverless Clusters. In Proceedings of the 31st International Symposium on High-Performance Parallel and Distributed Computing (Minneapolis, USA) (HPDC 2022). Association for Computing Machinery, New ...

    2022

  28. [35]

    Pablo Gimeno Sarroca and Marc Sánchez-Artigas. 2022. MLLess: Achieving Cost Efficiency in Serverless Machine Learning Training. arXiv e-prints (2022), arXiv–2206

    2022

  29. [36]

    Pawan Goyal, Harrick M Vin, and Haichen Cheng. 1997. Start-time fair queue- ing: A scheduling algorithm for integrated services packet switching networks. IEEE/ACM Transactions on networking 5, 5 (1997), 690–704

    1997

  30. [37]

    Jianfeng Gu, Yichao Zhu, Puxuan Wang, Mohak Chadha, and Michael Gerndt

  31. [38]

    Arpan Gujarati, Reza Karimi, Safya Alzayat, Wei Hao, Antoine Kaufmann, Ymir Vigfusson, and Jonathan Mace. 2020. Serving{DNNs} like clockwork: Perfor- mance predictability from the bottom up. In14th USENIX Symposium on Operating Systems Design and Implementation (OSDI 20) . 443–462

    2020

  32. [39]

    Mingcong Han, Hanze Zhang, Rong Chen, and Haibo Chen. 2022. Microsecond- scale preemption for concurrent{GPU-accelerated}{ DNN} inferences. In 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22) . 539–558

    2022

  33. [41]

    Cheol-Ho Hong, Ivor Spence, and Dimitrios S Nikolopoulos. 2017. GPU virtu- alization and scheduling methods: A comprehensive survey. ACM Computing Surveys (CSUR) 50, 3 (2017), 1–37

    2017

  34. [42]

    Ling-Hong Hung, Dimitar Kumanov, Xingzhi Niu, Wes Lloyd, and Ka Yee Yeung

  35. [43]

    2019.{Multi- Queue} Fair Queuing

    Mohammad Hedayati, Kai Shen, Michael L Scott, and Mike Marty. 2019.{Multi- Queue} Fair Queuing. In 2019 USENIX Annual Technical Conference (USENIX ATC 19). 301–314

    2019

  36. [44]

    Insu Jang, Adrian Tang, Taehoon Kim, Simha Sethumadhavan, and Jaehyuk Huh

  37. [45]

    Yankai Jiang, Rohan Basu Roy, Baolin Li, and Devesh Tiwari. 2024. EcoLife: Carbon-Aware Serverless Function Scheduling for Sustainable Computing. In SC24: International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE, 1–15

    2024

  38. [46]

    Aji John, Kristiina Ausmees, Kathleen Muenzen, Catherine Kuhn, and Amanda Tan. 2019. SWEEP: Accelerating Scientific Research Through Scalable Serverless Workflows. InProceedings of the 12th IEEE/ACM International Conference on Utility and Cloud Computing Companion - UCC ’19 Co...

    2019

  39. [47]

    Sitaram Iyer and Peter Druschel. 2001. Anticipatory scheduling: A disk scheduling framework to overcome deceptive idleness in synchronous I/O. In Proceedings of the eighteenth ACM symposium on Operating systems principles . 117–130

    2001

  40. [48]

    Kostis Kaffes, Neeraja J Yadwadkar, and Christos Kozyrakis. 2021. Practical scheduling for real-world serverless computing. arXiv preprint arXiv:2111.07226 (2021)

    2021 arXiv

  41. [49]

    InProceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems

    Heterogeneous isolated execution for commodity gpus. InProceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems . 455–468

  42. [50]

    Jaewook Kim, Tae Joon Jun, Daeyoun Kang, Dohyeun Kim, and Daeyoung Kim

  43. [51]

    Dimitar Kumanov, Ling-Hong Hung, Wes Lloyd, and Ka Yee Yeung. 2018. Server- less computing provides on-demand high performance computing for biomedical research. arXiv preprint arXiv:1807.11659 (2018)

    2018 arXiv

  44. [52]

    Justin San Juan and Bernard Wong. 2023. Reducing the Cost of GPU Cold Starts in Serverless Deep Learning Inference Serving. In 2023 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops). IEEE, Atlanta, ...

    2023

  45. [53]

    Ping-Min Lin and Alex Glikson. 2019. Mitigating Cold Starts in Serverless Platforms: A Pool-Based Approach. arXiv:1903.12221 [cs] (March 2019). http: //arxiv.org/abs/1903.12221 arXiv: 1903.12221

    2019 arXiv

  46. [54]

    Yadwadkar, and Christos Kozyrakis

    Kostis Kaffes, Neeraja J. Yadwadkar, and Christos Kozyrakis. 2022. Hermod: principled and practical scheduling for serverless functions. In Proceedings of the 13th Symposium on Cloud Computing . ACM, San Francisco California, 289–305. doi:10.1145/3542929.3563468

    2022

  47. [55]

    Kshiteej Mahajan, Arjun Balasubramanian, Arjun Singhvi, Shivaram Venkatara- man, Aditya Akella, Amar Phanishayee, and Shuchi Chawla. 2020. Themis: Fair and efficient{GPU} cluster scheduling. In 17th USENIX Symposium on Networked Systems Design and Implementation (NSDI 20) . 289–304

    2020

  48. [56]

    In 2018 26th Euromicro International Conference on Parallel, Distributed and Network-based Processing (PDP)

    GPU Enabled Serverless Computing Framework. In 2018 26th Euromicro International Conference on Parallel, Distributed and Network-based Processing (PDP). IEEE, Cambridge, 533–540. doi:10.1109/PDP2018.2018.00090

    2018

  49. [57]

    Zizhao Mo, Huanle Xu, and Wing Cheong Lau. 2024. Optimal Resource Efficiency with Fairness in Heterogeneous GPU Clusters. http://arxiv.org/abs/2403.18545 arXiv:2403.18545 [cs]

    2024 arXiv

  50. [58]

    Baolin Li, Tirthak Patel, Siddharth Samsi, Vijay Gadepally, and Devesh Tiwari

  51. [59]

    Diana M Naranjo, Sebastián Risco, Carlos de Alfonso, Alfonso Pérez, Ignacio Blanquer, and Germán Moltó. 2020. Accelerated serverless computing based on GPU virtualization. J. Parallel and Distrib. Comput. 139 (2020), 32–42

    2020

  52. [60]

    Kelvin KW Ng, Henri Maxime Demoulin, and Vincent Liu. 2023. Paella: Low- latency Model Serving with Software-defined GPU Scheduling. In Proceedings of the 29th Symposium on Operating Systems Principles . 595–610

    2023

  53. [61]

    Shutian Luo, Huanle Xu, Chengzhi Lu, Kejiang Ye, Guoyao Xu, Liping Zhang, Yu Ding, Jian He, and Chengzhong Xu. 2021. Characterizing microservice de- pendency and performance: Alibaba trace analysis. In Proceedings of the ACM Symposium on Cloud Computing . 412–426

    2021

  54. [62]

    Nvidia. 2023. NVIDIA Multi-Instance GPU User Guide. https://docs.nvidia.com/ datacenter/tesla/mig-user-guide/index.html

    2023

  55. [63]

    Johannes Manner, Martin EndreB, Tobias Heckel, and Guido Wirtz. 2018. Cold Start Influencing Factors in Function as a Service. In2018 IEEE/ACM International Conference on Utility and Cloud Computing Companion (UCC Companion) . IEEE, Zurich, 181–188. doi:10.1109/UCC-Companion.2...

    2018

  56. [64]

    Rajat Phull, Cheng-Hong Li, Kunal Rao, Hari Cadambi, and Srimat Chakradhar

  57. [65]

    Anup Mohan, Harshad Sane, Kshitij Doshi, Saikrishna Edupuganti, Vadim Sukhomlinov, and Naren Nayak. 2019. Agile Cold Starts for Scalable Server- less. USENIX Workshop on Hot Topics in Cloud Computing (HotCloud) (2019), 6

    2019

  58. [66]

    Francisco Romero, Mark Zhao, Neeraja J Yadwadkar, and Christos Kozyrakis

  59. [67]

    Rohan Basu Roy, Tirthak Patel, and Devesh Tiwari. 2022. Icebreaker: Warming serverless functions better with heterogeneity. In Proceedings of the 27th ACM International Conference on Architectural Support for Programming Languages and Operating Systems. 753–767

    2022

  60. [68]

    Nvidia. 2023. Nvidia MPS. https://docs.nvidia.com/deploy/mps/index.html

    2023

  61. [69]

    Klaus Satzke, Istemi Ekin Akkus, Ruichuan Chen, Ivica Rimac, Manuel Stein, Andre Beck, Paarijaat Aditya, Manohar Vanga, and Volker Hilt. 2020. Efficient GPU Sharing for Serverless Workflows. In Proceedings of the 1st Workshop on High Performance Serverless Computing . 17–24

    2020

  62. [70]

    Nathan Pemberton, Anton Zabreyko, Zhoujie Ding, Randy Katz, and Joseph Gonzalez. 2022. Kernel-as-a-Service: A Serverless Interface to GPUs. arXiv preprint arXiv:2212.08146 (2022)

    2022 arXiv

  63. [71]

    Mohammad Shahrad, Rodrigo Fonseca, Inigo Goiri, Gohar Chaudhry, Paul Ba- tum, Jason Cooke, Eduardo Laureano, Colby Tresness, Mark Russinovich, and Ricardo Bianchini. 2020. Serverless in the wild: Characterizing and optimizing the serverless workload at a large cloud provider. ...

    2020

  64. [72]

    Vaishaal Shankar, Karl Krauth, Kailas Vodrahalli, Qifan Pu, Benjamin Recht, Ion Stoica, Jonathan Ragan-Kelley, Eric Jonas, and Shivaram Venkataraman. 2020. Serverless linear algebra. In Proceedings of the 11th ACM Symposium on Cloud Computing. 281–295

    2020

  65. [73]

    Sashko Ristov, Christian Hollaus, and Mika Hautz. 2022. Colder than the warm start and warmer than the cold start! experience the spawn start in faas providers. In Proceedings of the 2022 Workshop on Advanced tools, programming languages, and PLatforms for Implementing and Eva...

    2022

  66. [74]

    Josef Spillner, Cristian Mateos, and David A. Monge. 2018. FaaSter, Better, Cheaper: The Prospect of Serverless Scientific Computing and HPC. In High Per- formance Computing, Esteban Mocskos and Sergio Nesmachnow (Eds.). Vol. 796. Springer International Publishing, Cham, 154–1...

    2018 doi

  67. [75]

    Aditya Sundarrajan, Mingdong Feng, Mangesh Kasbekar, and Ramesh K Sitara- man. 2017. Footprint descriptors: Theory and practice of cache provisioning in a global cdn. In Proceedings of the 13th International Conference on emerging Networking EXperiments and Technologies. 55–67

    2017

  68. [76]

    Amoghavarsha Suresh, Gagan Somashekar, Anandh Varadarajan, Veeren- dra Ramesh Kakarla, Hima Upadhyay, and Anshul Gandhi. 2020. ENSURE: Efficient Scheduling and Autonomous Resource Management in Serverless Envi- ronments. In 2020 IEEE International Conference on Autonomic Compu...

    2020

  69. [77]

    Amit Samanta and Ryan Stutsman. 2024. Fair, Efficient Multi-Resource Scheduling for Stateless Serverless Functions with Anubis. In 2024 IEEE 24th International Symposium on Cluster, Cloud and Internet Computing (CCGrid) . 106–112. doi:10. 1109/CCGrid59990.2024.00021

    2024

  70. [78]

    Dmitrii Ustiugov, Dohyun Park, Lazar Cvetković, Mihajlo Djokic, Hongyu Hè, Boris Grot, and Ana Klimovic. 2023. Enabling In-Vitro Serverless Systems Re- search. In Proceedings of the 4th Workshop on Resource Disaggregation and Server- less. 1–7. 13

    2023

  71. [79]

    Yadwadkar, Raluca Ada Popa, Joseph E

    Johann Schleier-Smith, Vikram Sreekanti, Anurag Khandelwal, Joao Carreira, Neeraja J. Yadwadkar, Raluca Ada Popa, Joseph E. Gonzalez, Ion Stoica, and David A. Patterson. 2021. What Serverless Computing is and Should Become: The next Phase of Cloud Computing. Commun. ACM 64, 5 ...

    2021 doi

  72. [80]

    Parichehr Vahidinia, Bahar Farahani, and Fereidoon Shams Aliee. 2022. Mitigating cold start problem in serverless computing: A reinforcement learning approach. IEEE Internet of Things Journal 10, 5 (2022), 3917–3927

    2022

  73. [81]

    Blesson Varghese, Javier Prades, Carlos Reano, and Federico Silla. 2015. Acceleration-as-a-service: Exploiting virtualised GPUs for a financial application. In 2015 IEEE 11th International Conference on e-Science . IEEE, 47–56

    2015

  74. [82]

    Ying Sheng, Shiyi Cao, Dacheng Li, Banghua Zhu, Zhuohan Li, Danyang Zhuo, Joseph E Gonzalez, and Ion Stoica. 2024. Fairness in serving large language models. In 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24). 965–988

    2024

  75. [83]

    Sebastian Werner, Jörn Kuhlenkamp, Markus Klems, Johannes Müller, and Stefan Tai. 2018. Serverless big data processing using matrix multiplication as example. In 2018 IEEE International Conference on Big Data (Big Data) . IEEE, 358–365

    2018

  76. [84]

    Bingyang Wu, Zili Zhang, Zhihao Bai, Xuanzhe Liu, and Xin Jin. 2023. Trans- parent GPU sharing in container clouds for deep learning workloads. In 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23) . 69–85

    2023

  77. [85]

    2024.{StreamBox}: A Lightweight{GPU}{ SandBox} for Serverless Inference Workflow

    Hao Wu, Yue Yu, Junxiao Deng, Shadi Ibrahim, Song Wu, Hao Fan, Ziyue Cheng, and Hai Jin. 2024.{StreamBox}: A Lightweight{GPU}{ SandBox} for Serverless Inference Workflow. In2024 USENIX Annual Technical Conference (USENIX ATC 24). 59–73

    2024

  78. [86]

    2014.{GPUvm}: Why Not Virtualizing{GPUs} at the Hypervisor?

    Yusuke Suzuki, Shinpei Kato, Hiroshi Yamada, and Kenji Kono. 2014.{GPUvm}: Why Not Virtualizing{GPUs} at the Hypervisor?. In 2014 USENIX Annual Tech- nical Conference (USENIX ATC 14) . 109–120

    2014

  79. [87]

    Fei Xu, Yiling Qin, Li Chen, Zhi Zhou, and Fangming Liu. 2021. 𝜆dnn: Achieving predictable distributed DNN training with serverless architectures. IEEE Trans. Comput. 71, 2 (2021), 450–463

    2021

  80. [88]

    Dmitrii Ustiugov, Plamen Petrov, Marios Kogias, Edouard Bugnion, and Boris Grot

  81. [89]

    In Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems

    Benchmarking, analysis, and optimization of serverless function snapshots. In Proceedings of the 26th ACM International Conference on Architectural Support for Programming Languages and Operating Systems . 559–572

  82. [90]

    Zhisheng Ye, Wei Gao, Qinghao Hu, Peng Sun, Xiaolin Wang, Yingwei Luo, Tianwei Zhang, and Yonggang Wen. 2024. Deep learning workload scheduling in gpu datacenters: A survey. Comput. Surveys 56, 6 (2024), 1–38

    2024

  83. [91]

    Ting-An Yeh, Hung-Hsin Chen, and Jerry Chou. 2020. KubeShare: A framework to manage GPUs as first-class and shared resources in container cloud. InProceedings of the 29th international symposium on high-performance parallel and distributed computing. 173–184

    2020

  84. [92]

    2022.{MLaaS} in the wild: Workload analysis and scheduling in{Large-Scale} heterogeneous{GPU} clusters

    Qizhen Weng, Wencong Xiao, Yinghao Yu, Wei Wang, Cheng Wang, Jian He, Yong Li, Liping Zhang, Wei Lin, and Yu Ding. 2022.{MLaaS} in the wild: Workload analysis and scheduling in{Large-Scale} heterogeneous{GPU} clusters. In 19th USENIX Symposium on Networked Systems Design and I...

    2022

  85. [93]

    Miao Zhang, Yifei Zhu, Cong Zhang, and Jiangchuan Liu. 2019. Video processing with serverless computing: A measurement study. InProceedings of the 29th ACM workshop on network and operating systems support for digital audio and video . 61–66

    2019

  86. [94]

    Han Zhao, Weihao Cui, Quan Chen, Shulai Zhang, Zijun Li, Jingwen Leng, Chao Li, Deze Zeng, and Minyi Guo. 2024. Towards Fast Setup and High Throughput of GPU Serverless Computing. http://arxiv.org/abs/2404.14691 arXiv:2404.14691 [cs]

    2024 arXiv

  87. [95]

    Yuxuan Zhao, Weikang Weng, Rob van Nieuwpoort, and Alexandru Uta. 2024. In Serverless, OS Scheduler Choice Costs Money: A Hybrid Scheduling Approach for Cheaper FaaS. In Proceedings of the 25th International Middleware Conference . ACM, Hong Kong Hong Kong, 172–184. doi:10.114...

    2024

  88. [96]

    Wencong Xiao, Romil Bhardwaj, Ramachandran Ramjee, Muthian Sivathanu, Nipun Kwatra, Zhenhua Han, Pratyush Patel, Xuan Peng, Hanyu Zhao, Quanlu Zhang, et al. 2018. Gandiva: Introspective cluster scheduling for deep learning. In 13th USENIX Symposium on Operating Systems Design ...

    2018

  89. [97]

    Paweł Zuk, Bartłomiej Przybylski, and Krzysztof Rzadca. 2022. Call Scheduling to Reduce Response Time of a FaaS System. In2022 IEEE International Conference on Cluster Computing (CLUSTER). 172–182. doi:10.1109/CLUSTER51413.2022.00031 ISSN: 2168-9253

    2022

  90. [98]

    Shinichi Yamagiwa and Koichi Wada. 2009. Performance study of interference on gpu and cpu resources with multiple applications. In 2009 IEEE International Symposium on Parallel & Distributed Processing . IEEE, 1–8

    2009

  91. [99]

    Yanan Yang, Laiping Zhao, Yiming Li, Huanyu Zhang, Jie Li, Mingyang Zhao, Xingzhen Chen, and Keqiu Li. 2022. INFless: a native serverless system for low- latency, high-throughput inference. In Proceedings of the 27th ACM International Conference on Architectural Support for Pr...

    2022

  92. [102]

    Hangchen Yu, Arthur M Peters, Amogh Akshintala, and Christopher J Rossbach

  93. [103]

    In Proceedings of the Workshop on Hot Topics in Operating Systems

    Automatic virtualization of accelerators. In Proceedings of the Workshop on Hot Topics in Operating Systems . 58–65

  94. [107]

    Jiafan Zhu, Xiao Zhang, Konstantinos Menychtas, Zhijing Gene Qin, Steven Hand, Dragos Sbirlea, and Yuang Liu. 2024. GASS: GPU Automated Sharing at Scale. In 2024 IEEE 17th International Conference on Cloud Computing (CLOUD) . 439–445. doi:10.1109/CLOUD62652.2024.00056 ISSN: 2159-6190

    2024

  95. [109]

    Pawel Zuk and Krzysztof Rzadca. 2020. Scheduling Methods to Reduce Response Latency of Function as a Service. In 2020 IEEE 32nd International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD) . 132–140. doi:10.1109/SBAC-PAD49847.2020.00028 ISSN: 2643...

    2020

  96. [2012]

    In Proceedings of the 21st international symposium on High-Performance Parallel and Distributed Computing

    Interference-driven resource management for GPU-based heterogeneous clusters. In Proceedings of the 21st international symposium on High-Performance Parallel and Distributed Computing . 109–120

  97. [2018]

    In Workshop on Systems for ML and Open Source Software at NeurIPS , Vol

    A case for serverless machine learning. In Workshop on Systems for ML and Open Source Software at NeurIPS , Vol. 2018

    2018

  98. [2019]

    bioRxiv (2019), 576199

    Rapid RNA sequencing data analysis using serverless computing. bioRxiv (2019), 576199

    2019

  99. [2021]

    In Proceedings of the ACM Symposium on Cloud Computing

    Llama: A heterogeneous & serverless framework for auto-tuning video analytics pipelines. In Proceedings of the ACM Symposium on Cloud Computing . 1–17

  100. [2022]

    In Proceedings of the 13th Symposium on Cloud Computing

    Miso: exploiting multi-instance gpu capability on multi-tenant gpu clusters. In Proceedings of the 13th Symposium on Cloud Computing . 173–189

  101. [2023]

    InProceedings of the Eighteenth European Conference on Computer Systems

    Palette load balancing: Locality hints for serverless functions. InProceedings of the Eighteenth European Conference on Computer Systems . 365–380

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.