{"id":"af2dddfe-b1f6-40f5-8775-4e99106550b7","arxiv_id":"2608.00742","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Kubernetes with per-VNI Slingshot isolation is demonstrated on ARM64 Isambard-AI for a Trusted Research Environment and multi-node LLM serving, with ~0.9 us latency overhead.","lead":"Multi-tenant Kubernetes on a Cray EX supercomputer is demonstrated by isolating Slingshot RDMA traffic per network namespace. The paper reports working deployments for a Trusted Research Environment and for distributed LLM serving, with a small latency overhead.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Privileged Slingshot workloads defeat the netns-based VNI tamper-resistance claim; multi-tenant security is unproven in the deployed configuration.","rationale":"The central contribution is a demonstration that Friese et al.'s namespace-scoped CXI/VNI isolation can be reproduced on IS-AI and applied to a TRE and distributed LLM serving. The security story depends entirely on the claim (Section 2.1) that a container cannot modify its network namespace, making the netns inode a tamper-resistant key for CXI service binding. Section 4.2, however, states that Slingshot workloads 'currently execute in a privileged namespace' and that this 'presents a broad attack surface and undermines the intent of the isolation model.' This is a direct contradiction between the deployment posture and the security precondition. A privileged pod has CAP_SYS_ADMIN and typical host filesystem access—sufficient in principle to enter another namespace or access CXI devices directly; the authors' own snooping test in Section 4.1 did not exercise this path. The load-bearing assumption therefore fails exactly where the paper wants to claim multi-tenant security. The performance results, while under-reported, are not the central vulnerability: even if the 3x TPS speedup were noise, the deployment would still be a functional demonstration; but if isolation is breakable, the 'secure multi-tenant' claim is void. The reader's weakest_assumption identifies the same issue, so I agree. The verdict remains conditional: the contribution should be accepted only with the security boundary either hardened (non-privileged Slingshot workloads) or explicitly withdrawn from the claim.","tokens_in":16210,"tokens_out":7058,"duration_ms":79873,"concrete_test":"Under the actual FRIDGE pod security standard (privileged), deploy a third 'attacker' pod that mounts the host root (e.g., hostPath /) and has CAP_SYS_ADMIN. From it, identify a victim pod's host PID via the mounted host /proc, run `nsenter -t <victim_pid> -n` to enter the victim's network namespace, then attempt to create a CXI RDMA endpoint using the victim's VNI and service ID (e.g., with fi_pingpong or libfabric). If endpoint creation succeeds or traffic is received, VNI isolation is not tamper-resistant. Also run `capsh --print` inside the current Slingshot workload to confirm whether CAP_SYS_ADMIN is present. A failure of the nsenter/endpoint creation would refute the concern.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim—that Kubernetes with Slingshot namespace-based VNI isolation enables secure multi-tenant AI/TRE workloads—depends on Section 2.1's assertion that network namespace IDs 'cannot be modified from within' containers, making CXI service/VNI binding tamper-resistant. That condition fails in the actual deployment: Section 4.2 states 'workloads requiring Slingshot access currently execute in a privileged namespace' and that this 'presents a broad attack surface and undermines the intent of the isolation model.' A privileged workload has CAP_SYS_ADMIN and can call setns() to join a victim's network namespace (e.g., via /proc/<pid>/ns/net), or directly access host CXI devices, invalidating the netns-inode boundary the isolation is built on. The paper's own penetration test (Section 4.1) only tried supplying a victim's service ID/VNI from an unprivileged container; it never tested the privileged case that FRIDGE actually uses. Therefore the multi-tenant security guarantee is unestablished for the demonstrated configuration.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper reports the deployment and evaluation of a multi-tenant Kubernetes service on Isambard-AI, an HPE Cray EX system with Slingshot interconnect and NVIDIA Grace-Hopper ARM64 nodes. The authors port Friese et al.'s namespace-based Slingshot RDMA/VNI isolation to ARM64, including a libfabric patch that supplies HSN netdev metadata via environment variables, and build two co-design use cases on top of it: a FRIDGE Satellite Trusted Research Environment and a KubeRay/Ray/vLLM distributed LLM serving stack. They evaluate point-to-point OSU benchmarks, a TRE workload submission path, and vLLM serving performance under four configurations. The paper claims that namespace-based VNI isolation adds no meaningful latency penalty, that the FRIDGE deployment is functional, and that the KubeRay/vLLM stack achieves roughly a 3x tokens-per-second improvement over the BriCS reference configuration, while acknowledging a number of operational and security limitations.","tokens_in":16484,"tokens_out":5962,"duration_ms":66021,"significance":"If the central claims held, this would be a valuable systems paper: it demonstrates portability of a cloud-native multi-tenant stack to a Slingshot-based national AI system, provides a concrete ARM64 port of the prior CXI/VNI isolation work, and documents an unusual combination of FRIDGE TRE and KubeRay/vLLM on bare-metal Kubernetes. The paper is refreshingly transparent about its limitations, including software-version details, public repositories (FRIDGE, the VNI service), and explicit future-work items. However, the security and performance claims are not yet supported with the necessary evidence: the deployed workloads run in a privileged namespace, which contradicts the tamper-resistance premise of the isolation model, and the aggregate results are reported without error bars or significance tests.","major_comments":[{"comment":"The central multi-tenant security claim is undermined by the paper's own deployment description. Section 2.1 asserts that network namespace IDs are governed from outside containers and 'cannot be modified from within,' making CXI service/VNI binding tamper-resistant. But Section 4.2 states that 'workloads requiring Slingshot access currently execute in a privileged namespace' and that this 'presents a broad attack surface and undermines the intent of the isolation model.' A privileged container has CAP_SYS_ADMIN and can call setns() into a victim's network namespace or directly access host CXI character devices, so the netns-inode boundary does not provide the claimed isolation in the configuration that is actually used by FRIDGE and the RayCluster stack. The penetration test in Section 4.1 only attempts snooping from an unprivileged third container; it does not test the privileged case.","section":"Section 2.1 / Section 4.2"},{"comment":"The text states that the two Kubernetes configurations express 'statistically identical' latency profiles converging to ~3.11 microseconds, and concludes that namespace-based VNI isolation adds no meaningful latency penalty. However, no error bars, confidence intervals, or significance tests are reported; the only methodological detail is 'we average 10 runs per test.' The load-bearing conclusion of zero meaningful latency overhead depends on this unsupported statistical claim. Please report per-configuration variance and perform an appropriate test (e.g., paired bootstrap or t-test) on the repeated runs, or weaken the claim accordingly.","section":"Section 4.1, Figure 6"},{"comment":"The headline serving results—mean ITL/TPOT 'less than half' and 'approximately a TPS speed-up of three times' for the vni: 'true' configuration—are reported without error bars or significance tests for the 10 runs per configuration. Additionally, the two baseline configurations (BriCS and the vLLM tutorial) are described only by external reference, so it is unclear whether differences in vLLM build flags, container images, NCCL/RDMA settings, or Ray cluster topology account for the observed speedup rather than the VNI isolation itself. Please provide exact configuration descriptions and statistical uncertainty for all four configurations, and clarify whether the comparison is apples-to-apples.","section":"Section 4.3, Figure 7"}],"minor_comments":[{"comment":"Typo: 'reduing' should be 'reducing'.","section":"Section 3.3"},{"comment":"Typo: 'P99 refers to he value' should be 'P99 refers to the value'.","section":"Figure 7 caption"},{"comment":"Typo: 'The current Slingshot isolation stack is function' should be 'is functional'.","section":"Section 5"},{"comment":"Inconsistent spelling: 'MetacontrollerDecaratorController' appears in text and Figure 4; should be 'DecoratorController'.","section":"Sections 3.2/3.3/Figure 4"},{"comment":"The term 'Triggered List Entry pool' (TLE) is used without definition; please expand the acronym or add it to the acronym list.","section":"Section 3.3"}],"recommendation":"major_revision","confidential_remarks":"This is an honest and useful co-design/experience paper, and the authors clearly identify many limitations themselves. The main blocker is that the headline claim of secure multi-tenant Slingshot isolation is not supported by the deployment as described: the paper's own Section 4.2 admits that workloads run in a privileged namespace, which invalidates the netns-based tamper-resistance argument. The performance claims also need statistical backing. I believe a revision that reframes the contribution as a baseline evaluation with explicit non-goals, and adds proper uncertainty quantification, would be publishable. I would not reject outright."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a genuinely useful systems paper that does what it says—reproduces and extends the Friese et al. Slingshot VNI-isolation work on ARM64, documents the patches, and shows two real use cases (FRIDGE TRE and vLLM serving). It is not a breakthrough, but it is honest and reproducible.\n\nThe new elements are concrete: the libfabric netdev lookup workaround with CXIP_DEFAULT_LINK/SPEED, the Metacontroller extension to Argo Workflows and RayClusters, the Mutating Admission Policy for RayCluster annotations, and the ARM64 build pipeline for Ray/vLLM. They ship commit hashes, a version table, and a public FRIDGE repo—credit where due. The OSU latency data (~3.11us vs ~2.22us) is plausible, and the conclusion that netns-VNI adds little overhead is reasonable, though calling two profiles 'statistically identical' without error bars is a stretch.\n\nThe soft spots are real but mostly acknowledged. The biggest: the multi-tenant security claim does not hold for the deployed configuration. The paper itself admits (Section 4.2) that Slingshot workloads run in a privileged namespace, which 'undermines the intent of the isolation model.' A privileged container could join another netns via setns() or touch CXI devices directly. The penetration test only tried the unprivileged case. So the paper's framing of 'secure computing' overstates what is demonstrated. The authors are candid, but the abstract and intro still lean on that word.\n\nSecond, the ~3x LLM TPS speedup looks like it is driven by configuration differences, not VNI isolation per se. The paper says they 'inadvertently optimized' their own stack; the BriCS/vLLM baselines may simply be less tuned. No error bars, no controlled ablations (their VNI-on vs VNI-off comparison is within their stack, but the cross-stack comparison is confounded). The claim that the stack 'does not degrade user experience' is supported, but the speedup claim needs a caveat.\n\nThird, the Cassini CXI service limit (3 per NIC) is a practical scalability bottleneck for the per-pod service model. They mention it but do not quantify the impact.\n\nBottom line: this is a paper for practitioners deploying Kubernetes on Slingshot/ARM64 systems. It deserves a serious referee—I would send it to review, mainly to force tighter performance methodology and to push the authors to either fix or more prominently caveat the security story. I would cite it if I were working on similar infrastructure.\n\nRecommendation: accept conditionally, with a request to add error bars, clarify baseline parity, and move the privileged-namespace limitation into the abstract.","headline":"Useful, honest systems engineering on ARM64 Kubernetes with Slingshot VNI isolation; the multi-tenant security claim is undercut by the paper's own admission of privileged namespaces, and the LLM speedup needs tighter baselines.","tokens_in":16973,"tokens_out":3053,"would_cite":true,"duration_ms":34138,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper shows Kubernetes can run as a multi-tenant service on an HPE Cray EX supercomputer with Slingshot interconnect, achieving per-container RDMA isolation at no latency cost.","keywords":["Kubernetes","Multi-tenancy","Slingshot","RDMA","VNI","Trusted Research Environments","Confidential Computing","Distributed LLMs"],"falsifier":"Run a Slingshot-capable container inside the privileged namespace and attempt to enter the network namespace of a different tenant's pod (for example via setns) or to open an RDMA endpoint using that tenant's CXI service ID and VNI; if either succeeds, the multi-tenant isolation claim is refuted.","tokens_in":1853,"feed_emoji":"⚡","tokens_out":2516,"duration_ms":83958,"temperature":0.7,"pith_summary":"The authors are trying to establish that Kubernetes can be a practical multi-tenant layer on a bare-metal HPC system with a high-speed Slingshot fabric, not as a replacement for Slurm but as a complementary service for AI, confidential computing, and governed data workloads. Their central demonstration is that per-container RDMA isolation via Virtual Network Identifier (VNI) domains can be reproduced on the ARM64 Isambard-AI platform and that this isolation adds no measurable latency penalty over ordinary Kubernetes networking. They support this with two co-designed use cases: a SATRE-compliant Trusted Research Environment for sensitive medical and health data, and a distributed, sandboxed, persistent LLM hosting service built from KubeRay, Ray, and vLLM. If correct, this work shows that cloud-native orchestration and bare-metal HPC fabric performance are compatible, opening national AI supercomputers to multi-tenant services with strong isolation.","feed_headline":"Slingshot HPC runs multi-tenant Kubernetes at full fabric speed","feed_subtitle":"Network-namespace VNI isolation adds no latency; sandboxed vLLM stack triples throughput over a reference setup.","key_machinery":"The load-bearing mechanism is Slingshot's Virtual Network Identifier (VNI) isolation, enforced by CXI services scoped to Linux network namespaces. A VNI is an integer label defining an isolated communication domain on the fabric, analogous to a VLAN, and Rosetta switches only route packets between endpoints in the same VNI domain. The paper follows the Friese et al. design: the CXI driver, CXI library, and libfabric are patched so that a CXI service is bound to the network namespace inode of a container; the CXI CNI plugin reads a vni annotation on the owning resource, creates a CXI service scoped to that namespace, and the VNI service manages VNI reservations as custom resources. Because ne","core_discovery":"The paper claims that the Slingshot namespace-based network isolation design of Friese et al. can be successfully reimplemented on Isambard-AI's ARM64 architecture, and that this VNI-scoped RDMA isolation adds no meaningful latency penalty compared to Kubernetes without it. Using OSU Micro-Benchmarks, the authors find that both Kubernetes configurations converge to about 3.11 microseconds for small-message latency, versus 2.22 microseconds for bare metal, while peak bandwidth reaches the Slingshot fabric's 200 Gb/s maximum. For the AI use case, a declarative KubeRay/Ray/vLLM deployment of Llama-3.1-405B-Instruct-FP8 in a tensor-parallel-4, pipeline-parallel-2 configuration achieves roughly 1","pith_inferences":["The paper's own admission that Slingshot-capable workloads currently run in a privileged pod security namespace means that the isolation stack, as deployed, presents a broad attack surface that undermines the intended tenant-isolation guarantee.","The roughly threefold TPS advantage over the BriCS and vLLM reference setups likely reflects configuration choices bundled with the stack (layer-split strategy, NCCL and RDMA settings, library versions) rather than VNI isolation alone; an ablation that varies one factor at a time would separate these effects.","A Kubernetes-native CXI Dynamic Resource Allocation driver, if it can mount HSN netdev devices and CXI character devices in a single resource claim, would remove the dependency on unmaintained device managers and the per-pod CXI service bottleneck, making the isolation model scale to many pods per node.","The same VNI-scoped isolation pattern should be testable on future GX-class Slingshot platforms and with other AI frameworks such as Kubeflow and KServe, once VNI assignment is generalized beyond the RayCluster annotation mechanism."],"forward_implications":["Kubernetes pods can use Slingshot RDMA with per-tenant fabric isolation at no detectable latency increase over ordinary Kubernetes networking, so cloud-native orchestration and HPC fabric performance are compatible on Cray EX systems.","A SATRE-compliant Trusted Research Environment can run on a national AI supercomputer, allowing governed research workloads to be dispatched to GPU nodes under a shared-responsibility model without data sharing agreements, using encrypted local storage and per-namespace network policies.","Persistent multi-node LLM serving with vLLM, Ray, and KubeRay in a VNI-isolated sandbox is user-viable: 120 ms TTFT and roughly 22 TPS at batch size one for a 405B-parameter model, with the VNI-enabled deployment achieving approximately three times the output TPS of the BriCS reference configuration.","The current per-pod CXI service allocation model creates a hard resource ceiling: each Cassini NIC exposes only three user Triggered List Entry pools, so removing the one-to-one mapping between VNI-enabled pods and CXI services is necessary before large-scale multi-tenant deployments are practical.","VNI exhaustion can be managed operationally by enforcing workflow deletion policies such as ttlStrategy, but production use will require lifecycle automation and scheduler visibility into VNI availability."],"supporting_citations":[{"why":"Friese et al. supply the namespace-scoped CXI/VNI isolation design and code that the paper ports to ARM64 and validates.","marker":"[6]"},{"why":"Isambard-AI platform paper grounds the description of the Cray EX supercomputer on which the deployment is evaluated.","marker":"[7]"},{"why":"FRIDGE repository supplies the Pulumi infrastructure-as-code for the Trusted Research Environment deployment.","marker":"[8]"},{"why":"Ray paper provides the distributed runtime framework that underpins the multi-node model hosting stack.","marker":"[10]"},{"why":"vLLM paper supplies the PagedAttention-based LLM serving engine whose performance is evaluated.","marker":"[11]"},{"why":"OSU Micro-Benchmarks are used to measure point-to-point bandwidth and latency across bare-metal and Kubernetes configurations.","marker":"[35]"},{"why":"Cerebras benchmark for Llama 3.1 405B supplies the TTFT comparison point of 240 ms.","marker":"[39]"},{"why":"Oracle MI300X benchmark supplies the TPS comparison point of 36 tokens per second for the same model.","marker":"[40]"},{"why":"vLLM bench framework supplies the workload simulation and metrics for ITL, TPOT, and output TPS.","marker":"[41]"},{"why":"BriCS distributed-vLLM guidelines provide one reference configuration that the VNI-enabled stack outperforms.","marker":"[44]"},{"why":"vLLM's Ray multi-node tutorial provides the other reference configuration used in the throughput comparison.","marker":"[45]"}],"fun_headline_variants":["Kubernetes on Slingshot: zero latency hit, full 200 Gb/s","AI serving on Kubernetes triples throughput with zero isolation overhead","Slingshot Kubernetes: VNI isolation at bare-metal latency","Multi-tenant K8s on Cray EX: no latency cost, full fabric speed","Kubernetes on HPC: Slingshot isolation adds zero latency, triples AI throughput"],"cache_read_input_tokens":18816,"weakest_assumption_plain":"The security isolation relies on the claim that network namespace IDs cannot be modified from inside a container, but Slingshot-capable workloads currently execute in a privileged Kubernetes namespace, so a compromised workload may have the host privileges needed to alter or bypass that binding.","fun_headline_variants_meta":{"raw":{"variants":["Kubernetes on Slingshot: zero latency hit, full 200 Gb/s","AI serving on Kubernetes triples throughput with zero isolation overhead","Slingshot Kubernetes: VNI isolation at bare-metal latency","Multi-tenant K8s on Cray EX: no latency cost, full fabric speed","Kubernetes on HPC: Slingshot isolation adds zero latency, triples AI throughput"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001,"raw_usage":{"total_tokens":4097,"prompt_tokens":800,"completion_tokens":3297,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":544,"completion_tokens_details":{"reasoning_tokens":3193}},"tokens_in":544,"tokens_out":3297,"duration_ms":26063,"temperature":1.0,"reasoning_tokens":3193,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T00:22:15.416389+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a Slingshot-capable container inside the privileged namespace and attempt to enter the network namespace of a different tenant's pod (for example via setns) or to open an RDMA endpoint using that tenant's CXI service ID and VNI; if either succeeds, the multi-tenant isolation claim is refuted.","supporting_citations":[{"cited_title":"Closing the HPC-Cloud Convergence Gap: Multi-Tenant Slingshot RDMA for Kubernetes","cited_arxiv_id":"2508.09663","evidence_quote":"Friese et al. supply the namespace-scoped CXI/VNI isolation design and code that the paper ports to ARM64 and validates."},{"cited_title":"Isambard-AI: a leadership-class supercomputer optimised specifically for artificial intelligence","cited_arxiv_id":null,"evidence_quote":"Isambard-AI platform paper grounds the description of the Cray EX supercomputer on which the deployment is evaluated."},{"cited_title":"FRIDGE on Isambard-AI, 2025","cited_arxiv_id":null,"evidence_quote":"FRIDGE repository supplies the Pulumi infrastructure-as-code for the Trusted Research Environment deployment."},{"cited_title":"Ray: A distributed framework for emerging AI applications","cited_arxiv_id":null,"evidence_quote":"Ray paper provides the distributed runtime framework that underpins the multi-node model hosting stack."},{"cited_title":"Efficient memory management for large language model serving with pagedattention","cited_arxiv_id":null,"evidence_quote":"vLLM paper supplies the PagedAttention-based LLM serving engine whose performance is evaluated."},{"cited_title":"OSU Micro-Benchmarks, 7.5","cited_arxiv_id":null,"evidence_quote":"OSU Micro-Benchmarks are used to measure point-to-point bandwidth and latency across bare-metal and Kubernetes configurations."},{"cited_title":"Llama 3.1 405b now runs at 969 tokens/s on cerebras infer- ence, 2024","cited_arxiv_id":null,"evidence_quote":"Cerebras benchmark for Llama 3.1 405B supplies the TTFT comparison point of 240 ms."},{"cited_title":"Serving llama 3.1 405b model with amd instinct mi300x ac- celerators, 2024","cited_arxiv_id":null,"evidence_quote":"Oracle MI300X benchmark supplies the TPS comparison point of 36 tokens per second for the same model."},{"cited_title":"vLLM: Benchmark CLI, 2026","cited_arxiv_id":null,"evidence_quote":"vLLM bench framework supplies the workload simulation and metrics for ITL, TPOT, and output TPS."},{"cited_title":"Distributed vllm inference, 2026","cited_arxiv_id":null,"evidence_quote":"BriCS distributed-vLLM guidelines provide one reference configuration that the VNI-enabled stack outperforms."},{"cited_title":"vllm parallelism and scaling, 2026","cited_arxiv_id":null,"evidence_quote":"vLLM's Ray multi-node tutorial provides the other reference configuration used in the throughput comparison."}],"review_version":1}