Pith. sign in

REVIEW 3 major objections 5 minor 49 references

The paper shows Kubernetes can run as a multi-tenant service on an HPE Cray EX supercomputer with Slingshot interconnect, achieving per-container RDMA isolation at no latency cost.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-05 00:22 UTC pith:BOEONJNJ

load-bearing objection Useful, honest systems engineering on ARM64 Kubernetes with Slingshot VNI isolation; the multi-tenant security claim is undercut by the paper's own admission of privileged namespaces, and the LLM speedup needs tighter baselines. the 3 major comments →

arxiv 2608.00742 v1 pith:BOEONJNJ submitted 2026-08-01 cs.DC cs.AIcs.PF

Multi-tenant Kubernetes Use Cases for AI, Secure Computing and Data Services, and More

Jake Watson , Sadaf R Alam , Christopher Woods , Abdelwahab Kawafi , Thomas Green , Ian Johnson , Ellis Pires , Jessica R. Jones
show 1 more author
Utz-Uwe Haus
This is my paper
classification cs.DC cs.AIcs.PF
keywords KubernetesMulti-tenancySlingshotRDMAVNITrusted Research EnvironmentsConfidential ComputingDistributed LLMs
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The authors are trying to establish that Kubernetes can be a practical multi-tenant layer on a bare-metal HPC system with a high-speed Slingshot fabric, not as a replacement for Slurm but as a complementary service for AI, confidential computing, and governed data workloads. Their central demonstration is that per-container RDMA isolation via Virtual Network Identifier (VNI) domains can be reproduced on the ARM64 Isambard-AI platform and that this isolation adds no measurable latency penalty over ordinary Kubernetes networking. They support this with two co-designed use cases: a SATRE-compliant Trusted Research Environment for sensitive medical and health data, and a distributed, sandboxed, persistent LLM hosting service built from KubeRay, Ray, and vLLM. If correct, this work shows that cloud-native orchestration and bare-metal HPC fabric performance are compatible, opening national AI supercomputers to multi-tenant services with strong isolation.

Core claim

The paper claims that the Slingshot namespace-based network isolation design of Friese et al. can be successfully reimplemented on Isambard-AI's ARM64 architecture, and that this VNI-scoped RDMA isolation adds no meaningful latency penalty compared to Kubernetes without it. Using OSU Micro-Benchmarks, the authors find that both Kubernetes configurations converge to about 3.11 microseconds for small-message latency, versus 2.22 microseconds for bare metal, while peak bandwidth reaches the Slingshot fabric's 200 Gb/s maximum. For the AI use case, a declarative KubeRay/Ray/vLLM deployment of Llama-3.1-405B-Instruct-FP8 in a tensor-parallel-4, pipeline-parallel-2 configuration achieves roughly 1

What carries the argument

The load-bearing mechanism is Slingshot's Virtual Network Identifier (VNI) isolation, enforced by CXI services scoped to Linux network namespaces. A VNI is an integer label defining an isolated communication domain on the fabric, analogous to a VLAN, and Rosetta switches only route packets between endpoints in the same VNI domain. The paper follows the Friese et al. design: the CXI driver, CXI library, and libfabric are patched so that a CXI service is bound to the network namespace inode of a container; the CXI CNI plugin reads a vni annotation on the owning resource, creates a CXI service scoped to that namespace, and the VNI service manages VNI reservations as custom resources. Because ne

Load-bearing premise

The security isolation relies on the claim that network namespace IDs cannot be modified from inside a container, but Slingshot-capable workloads currently execute in a privileged Kubernetes namespace, so a compromised workload may have the host privileges needed to alter or bypass that binding.

What would settle it

Run a Slingshot-capable container inside the privileged namespace and attempt to enter the network namespace of a different tenant's pod (for example via setns) or to open an RDMA endpoint using that tenant's CXI service ID and VNI; if either succeeds, the multi-tenant isolation claim is refuted.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Kubernetes pods can use Slingshot RDMA with per-tenant fabric isolation at no detectable latency increase over ordinary Kubernetes networking, so cloud-native orchestration and HPC fabric performance are compatible on Cray EX systems.
  • A SATRE-compliant Trusted Research Environment can run on a national AI supercomputer, allowing governed research workloads to be dispatched to GPU nodes under a shared-responsibility model without data sharing agreements, using encrypted local storage and per-namespace network policies.
  • Persistent multi-node LLM serving with vLLM, Ray, and KubeRay in a VNI-isolated sandbox is user-viable: 120 ms TTFT and roughly 22 TPS at batch size one for a 405B-parameter model, with the VNI-enabled deployment achieving approximately three times the output TPS of the BriCS reference configuration.
  • The current per-pod CXI service allocation model creates a hard resource ceiling: each Cassini NIC exposes only three user Triggered List Entry pools, so removing the one-to-one mapping between VNI-enabled pods and CXI services is necessary before large-scale multi-tenant deployments are practical.
  • VNI exhaustion can be managed operationally by enforcing workflow deletion policies such as ttlStrategy, but production use will require lifecycle automation and scheduler visibility into VNI availability.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper's own admission that Slingshot-capable workloads currently run in a privileged pod security namespace means that the isolation stack, as deployed, presents a broad attack surface that undermines the intended tenant-isolation guarantee.
  • The roughly threefold TPS advantage over the BriCS and vLLM reference setups likely reflects configuration choices bundled with the stack (layer-split strategy, NCCL and RDMA settings, library versions) rather than VNI isolation alone; an ablation that varies one factor at a time would separate these effects.
  • A Kubernetes-native CXI Dynamic Resource Allocation driver, if it can mount HSN netdev devices and CXI character devices in a single resource claim, would remove the dependency on unmaintained device managers and the per-pod CXI service bottleneck, making the isolation model scale to many pods per node.
  • The same VNI-scoped isolation pattern should be testable on future GX-class Slingshot platforms and with other AI frameworks such as Kubeflow and KServe, once VNI assignment is generalized beyond the RayCluster annotation mechanism.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper reports the deployment and evaluation of a multi-tenant Kubernetes service on Isambard-AI, an HPE Cray EX system with Slingshot interconnect and NVIDIA Grace-Hopper ARM64 nodes. The authors port Friese et al.'s namespace-based Slingshot RDMA/VNI isolation to ARM64, including a libfabric patch that supplies HSN netdev metadata via environment variables, and build two co-design use cases on top of it: a FRIDGE Satellite Trusted Research Environment and a KubeRay/Ray/vLLM distributed LLM serving stack. They evaluate point-to-point OSU benchmarks, a TRE workload submission path, and vLLM serving performance under four configurations. The paper claims that namespace-based VNI isolation adds no meaningful latency penalty, that the FRIDGE deployment is functional, and that the KubeRay/vLLM stack achieves roughly a 3x tokens-per-second improvement over the BriCS reference configuration, while acknowledging a number of operational and security limitations.

Significance. If the central claims held, this would be a valuable systems paper: it demonstrates portability of a cloud-native multi-tenant stack to a Slingshot-based national AI system, provides a concrete ARM64 port of the prior CXI/VNI isolation work, and documents an unusual combination of FRIDGE TRE and KubeRay/vLLM on bare-metal Kubernetes. The paper is refreshingly transparent about its limitations, including software-version details, public repositories (FRIDGE, the VNI service), and explicit future-work items. However, the security and performance claims are not yet supported with the necessary evidence: the deployed workloads run in a privileged namespace, which contradicts the tamper-resistance premise of the isolation model, and the aggregate results are reported without error bars or significance tests.

major comments (3)
  1. [Section 2.1 / Section 4.2] The central multi-tenant security claim is undermined by the paper's own deployment description. Section 2.1 asserts that network namespace IDs are governed from outside containers and 'cannot be modified from within,' making CXI service/VNI binding tamper-resistant. But Section 4.2 states that 'workloads requiring Slingshot access currently execute in a privileged namespace' and that this 'presents a broad attack surface and undermines the intent of the isolation model.' A privileged container has CAP_SYS_ADMIN and can call setns() into a victim's network namespace or directly access host CXI character devices, so the netns-inode boundary does not provide the claimed isolation in the configuration that is actually used by FRIDGE and the RayCluster stack. The penetration test in Section 4.1 only attempts snooping from an unprivileged third container; it does not test the privileged case.
  2. [Section 4.1, Figure 6] The text states that the two Kubernetes configurations express 'statistically identical' latency profiles converging to ~3.11 microseconds, and concludes that namespace-based VNI isolation adds no meaningful latency penalty. However, no error bars, confidence intervals, or significance tests are reported; the only methodological detail is 'we average 10 runs per test.' The load-bearing conclusion of zero meaningful latency overhead depends on this unsupported statistical claim. Please report per-configuration variance and perform an appropriate test (e.g., paired bootstrap or t-test) on the repeated runs, or weaken the claim accordingly.
  3. [Section 4.3, Figure 7] The headline serving results—mean ITL/TPOT 'less than half' and 'approximately a TPS speed-up of three times' for the vni: 'true' configuration—are reported without error bars or significance tests for the 10 runs per configuration. Additionally, the two baseline configurations (BriCS and the vLLM tutorial) are described only by external reference, so it is unclear whether differences in vLLM build flags, container images, NCCL/RDMA settings, or Ray cluster topology account for the observed speedup rather than the VNI isolation itself. Please provide exact configuration descriptions and statistical uncertainty for all four configurations, and clarify whether the comparison is apples-to-apples.
minor comments (5)
  1. [Section 3.3] Typo: 'reduing' should be 'reducing'.
  2. [Figure 7 caption] Typo: 'P99 refers to he value' should be 'P99 refers to the value'.
  3. [Section 5] Typo: 'The current Slingshot isolation stack is function' should be 'is functional'.
  4. [Sections 3.2/3.3/Figure 4] Inconsistent spelling: 'MetacontrollerDecaratorController' appears in text and Figure 4; should be 'DecoratorController'.
  5. [Section 3.3] The term 'Triggered List Entry pool' (TLE) is used without definition; please expand the acronym or add it to the acronym list.

Circularity Check

0 steps flagged

No significant circularity: claims are empirical reproductions validated by external benchmarks; cited prior work is a starting point, not a fitted input, and the privileged-namespace caveat is an acknowledged limitation rather than a circular step.

full rationale

This paper is an empirical systems/evaluation paper rather than a derivation. It ports the Slingshot namespace-based VNI isolation implementation of Friese et al. to ARM64, deploys FRIDGE and a KubeRay/Ray/vLLM stack, and measures performance. There are no equations that reduce to their inputs, no fitted parameters renamed as predictions, and no uniqueness theorem imported from the authors' prior work. The only potential circularity concern is that Friese et al. [6] includes co-author Utz-Uwe Haus and the FRIDGE repository [8] is by co-author Jake Watson; however, the paper does not rely on these as unverified assumptions. It reproduces the cited implementation and validates it with fi_pingpong connectivity/isolation tests and OSU micro-benchmarks, which would fail if the cited implementation were incorrect. The vLLM/TPS comparisons use independently specified reference configurations (BriCS guidelines, vLLM tutorial) and an external dataset, so the claimed ~3x TPS speedup is not constructed from the paper's own parameters. The paper also explicitly acknowledges the main weakness: Section 4.2 states 'workloads requiring Slingshot access currently execute in a privileged namespace. This presents a broad attack surface and undermines the intent of the isolation model of the cluster.' This is a stated limitation that weakens the security claim, but it is not circular reasoning. Similarly, the libfabric netdev_lookup patch is presented as a pragmatic workaround, not as a prediction. Overall, the central claims have independent empirical content and are not forced by self-citation or definition. Score 1 reflects the presence of minor self-citations that are not load-bearing.

Axiom & Free-Parameter Ledger

2 free parameters · 4 axioms · 0 invented entities

The paper's central claims rest on the correctness of the prior VNI isolation approach, the tamper-resistance of network namespaces, the fairness of benchmark baselines, and the sufficiency of the libfabric environment variable workaround. No parameters are fitted to data; the hardcoded CXI speed/link values are the closest thing to ad hoc constants.

free parameters (2)
  • CXIP_DEFAULT_LINK = 1
    Hardcoded link-up state used by the patched libfabric netdev_lookup() to bypass missing HSN netdev sysfs entries inside containers (Section 3.1). Chosen to match physical link state, not fitted to data.
  • CXIP_DEFAULT_SPEED = 200000
    Hardcoded interface speed in Mb/s supplied through the same libfabric patch; matches the 200Gb/s Slingshot port speed (Section 3.1).
axioms (4)
  • domain assumption Slingshot VNI enforcement in Rosetta switches and CXI service authorization works as described in Section 2.1
    The entire multi-tenancy story rests on the switch dropping cross-VNI packets and CXI endpoint creation failing without the correct (svc_id, vni) tuple. Adopted from Friese et al. without re-verification in this paper.
  • domain assumption Network namespace IDs cannot be modified from within a container
    Section 2.1 claims netns IDs are governed from outside, providing a tamper-resistant CXI service binding. This is later undermined by the admission that Slingshot workloads run in privileged namespaces (Section 4.2).
  • domain assumption The BriCS and vLLM tutorial configurations are fair, comparably optimized baselines
    The reported performance advantage (up to 3x TPS) depends on the baselines using equivalent parallelism, NCCL/RDMA settings, and vLLM parameters. The paper does not document these in enough detail to rule out configuration mismatch.
  • ad hoc to paper The libfabric patch environment variables fully substitute for HSN netdev sysfs metadata
    Section 3.1 hardcodes link state and speed to work around missing device mounts; correctness relies on these constants being sufficient for CXI provider initialization across all workloads.

pith-pipeline@v1.3.0-alltime-deepseek · 15973 in / 12480 out tokens · 126189 ms · 2026-08-05T00:22:15.416389+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Multi-tenant Kubernetes Use Cases for AI, Secure Computing and Data Services, and More." pith.science (2026). https://pith.science/paper/BOEONJNJ

@misc{pith2026260800742,
  author       = {Pith},
  title        = {Pith review of: Multi-tenant Kubernetes Use Cases for AI, Secure Computing and Data Services, and More},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BOEONJNJ}},
  note         = {Machine review of arXiv:2608.00742}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Kubernetes, as a container orchestration engine, has been widely used in cloud-native ecosystems for several years. In supercomputing ecosystems, especially where bare-metal performance for compute and network devices are considered, the adoption is somewhat limited. However, with the increasing diversity of use cases such as AI, secure and confidential computing for sensitive data, and mixed workload orchestration, a traditional, single-tenant batch computing system does not offer the flexibility and reproducibility to which public cloud users are accustomed. Note that Kubernetes is not considered a replacement for batch scheduling systems, which have powerful features for large-scale MPI jobs with thousands of network end points. Rather, it is a complementary service provided as part of a national AI Research Resource. We evaluate Kubernetes deployment on a Hewlett Packard Enterprise (HPE) Cray EX supercomputerwith HPE Slingshot interconnect, called Isambard-AI, with co-design use cases. One is a Trusted Research Environment used for medical and health sciences. The other combines KubeRay, Ray, and vLLM to provide a distributed, sandboxed, persistent AI model hosting service targeting multi-tenant confidential computing. We discuss challenges and lessons learned, and where further development is needed to offer a production Kubernetes-as-a-Service on HPE Cray EX (and later) platforms.

Figures

Figures reproduced from arXiv: 2608.00742 by Abdelwahab Kawafi, Christopher Woods, Ellis Pires, Ian Johnson, Jake Watson, Jessica R. Jones, Sadaf R Alam, Thomas Green, Utz-Uwe Haus.

Figure 1
Figure 1. Figure 1: The TRE architecture developed by the FRIDGE project team from The Alan Turing Institute, University Col￾lege London, University of Bristol, and University of Cam￾bridge for public-cloud, on-premises and AIRR supercomput￾ers including Isambard-AI at Bristol and Dawn at Cambridge. Figure reproduced from Alan Turing Institute [14]. SATRE [9] is a compliance framework for TREs developed by Data and Analytics … view at source ↗
Figure 2
Figure 2. Figure 2: A layered architecture stack for AI infrastructure, comprising (bottom to top): a Compute Substrate of GPUs and CPUs; a Container Orchestrator, primarily Kubernetes; a Distributed Compute Engine; a Training & Inference Frame￾work spanning transformer-focused and general distributed training libraries; and an AI Workload layer. Figure repro￾duced from Anyscale [17]. Building a modern inference platform for … view at source ↗
Figure 3
Figure 3. Figure 3: represents a minimal Kubernetes manifest con￾taining the necessary modifications for the Per-Resource VNI allocation model, including the VNI annotation, CXI device mount, host library mounts, and environment variables. 3.2 Trusted Research Environments We deploy a FRIDGE instance on IS-AI using the Python im￾plementation of Pulumi infrastructure-as-code [27], which is publicly available [8]. The Pulumi co… view at source ↗
Figure 4
Figure 4. Figure 4: Modifications to the Metacontroller DecaratorCon￾troller of the VNI service that enable it to listen for Argo Workflow and RayCluster creation events for the purposes of VNI allocation. library mounts required for CXI access. This is a broad grant and we explore less intrusive solutions in Section 5. 3.3 Secure and Multi-node Model Hosting We deploy the KubeRay, Ray, and vLLM stack to show the multi-node A… view at source ↗
Figure 5
Figure 5. Figure 5: Mutating Admission Policy code to annotate Ray￾Cluster resources with vni: "true" in order for the Ray￾Cluster to receive VNI allocations. (MAP) [34] (see [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 7
Figure 7. Figure 7: Benchmarking results on the ShareGPT_Vicuna_unfiltered dataset for the Meta-Llama-3.1-405B-Instruct-FP8 model deployed in the four infrastructure setups. (a) and (b) show percentile performance for the ITL and TPOT metrics, a lower time in milliseconds (ms) is better. P99 refers to he value below which 99% of requests fall, it reveals worst-case performance for the slowest 1% of requests. (c) shows the rel… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

49 extracted references · 48 canonical work pages · 1 internal anchor

  1. [1]

    Slinky: SchedMD’s set of projects to enable interoperability between Slurm and Kubernetes, 2026

    SchedMD. Slinky: SchedMD’s set of projects to enable interoperability between Slurm and Kubernetes, 2026. URLhttps://slurm.schedmd. com/slinky.html. [Online; accessed March 16, 2026]

    work page 2026

  2. [2]

    Kubernetes Operator for Slurm Clusters, 2026

    SchedMD. Kubernetes Operator for Slurm Clusters, 2026. URLhttps:// slinky.schedmd.com/projects/slurm-operator/en/release-1.0/. [Online; accessed March 16, 2026]

    work page 2026

  3. [3]

    Batch System Initiative Working Group, 2026

    CNCF. Batch System Initiative Working Group, 2026. URLhttps://tag- runtime.cncf.io/wgs/bsi/. [Online; accessed March 16, 2026]

    work page 2026

  4. [4]

    Cloud-native Container Batch Scheduler for HPC Workloads,

    Volcano. Cloud-native Container Batch Scheduler for HPC Workloads,

    work page

  5. [5]

    Multi-Kubernetes Cluster Batch Job Meta-Scheduler, 2026

    Armada. Multi-Kubernetes Cluster Batch Job Meta-Scheduler, 2026. URLhttps://armadaproject.io/. [Online; accessed March 16, 2026]

    work page 2026

  6. [6]

    Closing the HPC-Cloud Convergence Gap: Multi-Tenant Slingshot RDMA for Kubernetes

    Philipp A. Friese, Ahmed Eleliemy, Utz-Uwe Haus, and Martin Schulz. Closing the hpc-cloud convergence gap: Multi-tenant slingshot rdma for kubernetes, 2025. URLhttps://arxiv.org/abs/2508.09663

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  7. [7]

    Isambard-AI: a leadership-class supercomputer optimised specifically for artificial intelligence

    Simon McIntosh-Smith, Sadaf Alam, and Christopher Woods. Isambard-AI: a leadership-class supercomputer optimised specifically for artificial intelligence. InProceedings of the Cray User Group, pages 44–54. ACM, 2024

    work page 2024

  8. [8]

    FRIDGE on Isambard-AI, 2025

    Jake Watson. FRIDGE on Isambard-AI, 2025. URLhttps://github.com/ isambard-sc/fridge

    work page 2025

  9. [9]

    SATRE, 2026

    SATRE. SATRE, 2026. URLhttps://satre-specification.readthedocs.io/ en/stable/

    work page 2026

  10. [10]

    Ray: A distributed framework for emerging AI applications

    Philipp Moritzet al. Ray: A distributed framework for emerging AI applications. In13th USENIX Symposium on Operating Systems Design and Implementation (OSDI 18), pages 561–577. USENIX Association, 2018

    work page 2018

  11. [11]

    Efficient memory management for large language model serving with pagedattention

    Woosuk Kwonet al. Efficient memory management for large language model serving with pagedattention. InProceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023

    work page 2023

  12. [12]

    Dynamic Resource Allocation, 2026

    Kubernetes. Dynamic Resource Allocation, 2026. URL https://kubernetes.io/docs/concepts/scheduling-eviction/dynamic- resource-allocation/. [Online; accessed March 20, 2026]. CUG ’26, April 26-30, 2026, Nice, France Bristol Centre for Supercomputing (BriCS) and HPE

    work page 2026

  13. [13]

    Rancher kubernetes engine 2, 2026

    SUSE Rancher. Rancher kubernetes engine 2, 2026. URLhttps://docs. rke2.io/. [Online; accessed March 30, 2026]

    work page 2026

  14. [14]

    FRIDGE, 2026

    Alan Turing Institute. FRIDGE, 2026. URLhttps://github.com/alan- turing-institute/fridge/blob/main/docs/architecture/architecture. md. [Online; accessed March 20, 2026]

    work page 2026

  15. [15]

    DARE UK, 2026

    DARE UK. DARE UK, 2026. URLhttps://dareuk.org.uk/

    work page 2026

  16. [16]

    FRIDGE, 2026

    FRIDGE. FRIDGE, 2026. URLhttps://dareuk.org.uk/how-we-work/ ongoing-activities/dare-uk-early-adopters/fridge/

    work page 2026

  17. [17]

    Anyscale, 2026

    Anyscale. Anyscale, 2026. URLhttps://www.anyscale.com/blog/ai- compute-open-source-stack-kubernetes-ray-pytorch-vllm. [Online; accessed March 23, 2026]

    work page 2026

  18. [18]

    Designing a kubernetes operator for machine learning applications

    Ali Kansoet al. Designing a kubernetes operator for machine learning applications. InProceedings of the Seventh International Workshop on Container Technologies and Container Clouds, 2021. URLhttps: //api.semanticscholar.org/CorpusID:244663710

    work page 2021

  19. [19]

    Huggingface’s transformers: State-of-the-art nat- ural language processing.arXiv preprint arXiv:1910.03771, 2019

    Thomas Wolfet al. Huggingface’s transformers: State-of-the-art nat- ural language processing.arXiv preprint arXiv:1910.03771, 2019. doi: 10.48550/arXiv.1910.03771. URLhttps://arxiv.org/abs/1910.03771

    Pith/arXiv arXiv doi:10.48550/arxiv.1910.03771 1910

  20. [20]

    Longhorn, 2026

    Longhorn. Longhorn, 2026. URLhttps://longhorn.io/. [Online; accessed March 24, 2026]

    work page 2026

  21. [21]

    NVIDIA GPU Operator, 2026

    NVIDIA. NVIDIA GPU Operator, 2026. URLhttps://docs.nvidia.com/ datacenter/cloud-native/gpu-operator/latest/overview.html/. [Online; accessed March 24, 2026]

    work page 2026

  22. [22]

    KubeRay Operator, 2026

    Ray Project. KubeRay Operator, 2026. URLhttps://github.com/ray- project/kuberay. [Online; accessed March 30, 2026]

    work page 2026

  23. [23]

    Metacontroller, 2026

    Metacontroller. Metacontroller, 2026. URLhttps://github.com/ metacontroller/metacontroller. [Online; accessed March 30, 2026]

    work page 2026

  24. [24]

    SmarterDeviceManager, 2026

    SmarterDeviceManager. SmarterDeviceManager, 2026. URLhttps: //github.com/smarter-project/smarter-device-manager

    work page 2026

  25. [25]

    HPE CXI Kubernetes Device Plugin, 2026

    HPE CXI Kubernetes Device Plugin. HPE CXI Kubernetes Device Plugin, 2026. URLhttps://github.com/HewlettPackard/cxi-k8s-device- plugin

    work page 2026

  26. [26]

    Akri, 2026

    Akri. Akri, 2026. URLhttps://docs.akri.sh/

    work page 2026

  27. [27]

    Pulumi Infrastructure-as-Code, 2026

    Pulumi. Pulumi Infrastructure-as-Code, 2026. URLhttps://www. pulumi.com/

    work page 2026

  28. [28]

    Metallb, 2026

    MetalLB. Metallb, 2026. URLhttps://metallb.io/. [Online; accessed March 23, 2026]

    work page 2026

  29. [29]

    Cilium, 2026

    Cilium. Cilium, 2026. URLhttps://cilium.io/. [Online; accessed March 23, 2026]

    work page 2026

  30. [30]

    Argo, 2026

    Argo. Argo, 2026. URLhttps://argoproj.github.io/workflows/. [Online; accessed March 23, 2026]

    work page 2026

  31. [31]

    Ray Project Docker, 2026

    Ray Project. Ray Project Docker, 2026. URLhttps://hub.docker.com/ layers/rayproject/ray/2.54.0-py312-cu126-aarch64/images/sha256- c64c52747e1540949f2340d1bee36c062a964123d7d601a3f803333e1bb2d059. [Online; accessed March 24, 2026]

    work page 2026

  32. [32]

    Spack, 2026

    Spack. Spack, 2026. URLhttps://spack.io/. [Online; accessed March 24, 2026]

    work page 2026

  33. [33]

    Collette, Gregory L

    Todd Gamblin, Matthew LeGendre, Michael R. Collette, Gregory L. Lee, Adam Moody, Bronis R. de Supinski, and Scott Futral. The spack package manager: bringing order to hpc software chaos. InProceed- ings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC ’15, New York, NY, USA, 2015. Association for Computi...

    work page arXiv 2015

  34. [34]

    Mutating Admission Policy, 2026

    Kubernetes. Mutating Admission Policy, 2026. URL https://kubernetes.io/docs/reference/access-authn-authz/mutating- admission-policy/. [Online; accessed March 20, 2026]

    work page 2026

  35. [35]

    OSU Micro-Benchmarks, 7.5

    OSU. OSU Micro-Benchmarks, 7.5. URLhttps://mvapich.cse.ohio- state.edu/benchmarks/

    work page

  36. [36]

    Kyverno, 2026

    Kyverno. Kyverno, 2026. URLhttps://kyverno.io/. [Online; accessed March 23, 2026]

    work page 2026

  37. [37]

    KubeArmor, 2026

    KubeArmor. KubeArmor, 2026. URLhttps://kubearmor.io/. [Online; accessed March 23, 2026]

    work page 2026

  38. [38]

    Key metrics for LLM inference, 2026

    BentoML. Key metrics for LLM inference, 2026. URLhttps://bentoml. com/llm/inference-optimization/llm-inference-metrics. [Online; ac- cessed March 12, 2026]

    work page 2026

  39. [39]

    Llama 3.1 405b now runs at 969 tokens/s on cerebras infer- ence, 2024

    Cerebras. Llama 3.1 405b now runs at 969 tokens/s on cerebras infer- ence, 2024. URLhttps://www.cerebras.ai/blog/llama-405b-inference

    work page 2024

  40. [40]

    Serving llama 3.1 405b model with amd instinct mi300x ac- celerators, 2024

    Oracle. Serving llama 3.1 405b model with amd instinct mi300x ac- celerators, 2024. URLhttps://blogs.oracle.com/cloud-infrastructure/ serving-llama-31-405b-model-with-amd-mi300x-gpus

    work page 2024

  41. [41]

    vLLM: Benchmark CLI, 2026

    vLLM. vLLM: Benchmark CLI, 2026. URLhttps://docs.vllm.ai/en/ latest/benchmarking/cli/. [Online; accessed March 13, 2026]

    work page 2026

  42. [42]

    Sharegpt_vicuna_unfiltered, 2023

    anon8231489123. Sharegpt_vicuna_unfiltered, 2023. URL https://huggingface.co/datasets/anon8231489123/ShareGPT_ Vicuna_unfiltered. Accessed: 2026-04-17

    work page 2023

  43. [43]

    Meta-llama-3.1-405b-instruct, 2026

    Meta Llama. Meta-llama-3.1-405b-instruct, 2026. URLhttps:// huggingface.co/meta-llama/Llama-3.1-405B-Instruct

    work page 2026

  44. [44]

    Distributed vllm inference, 2026

    BriCS. Distributed vllm inference, 2026. URLhttps://docs.isambard. ac.uk/user-documentation/tutorials/distributed-inference/. Accessed: 2026-04-20

    work page 2026

  45. [45]

    vllm parallelism and scaling, 2026

    vLLM Authors. vllm parallelism and scaling, 2026. URLhttps://docs. vllm.ai/en/stable/serving/parallelism_scaling/. Accessed: 2026-04-20

    work page 2026

  46. [46]

    USS – User Services Software, 2026

    USS. USS – User Services Software, 2026. URLhttps: //support.hpe.com/hpesc/public/docDisplay?docId=dp00005611en_ us&page=install/About_USS.html&docLocale=en_US

    work page 2026

  47. [47]

    Kubeflow.https://www.kubeflow.org/, 2026

    The Kubeflow Authors. Kubeflow.https://www.kubeflow.org/, 2026. [Online; accessed 20-April-2026]

    work page 2026

  48. [48]

    KServe.https://kserve.github.io/website/, 2026

    The KServe Authors. KServe.https://kserve.github.io/website/, 2026. [Online; accessed 20-April-2026]. Acronyms AIRRAI Research Resource AISIAI Security Institute APIApplication Programming Interface ARPAddress Resolution Protocol BGPBorder Gateway Protocol BriCSBristol Centre for Supercomputing CFSCray Framework Service CNCFCloud Native Computing Foundati...

    work page 2026

  49. [2026]

    [Online; accessed March 16, 2026]

    URLhttps://volcano.sh/en/docs/. [Online; accessed March 16, 2026]

    work page 2026