{"id":"a82ea0ee-0959-4723-9312-9fa8fb2b9ad0","arxiv_id":"2508.20274","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A VM-deployable controller combining dynamic MIG, PCIe-aware placement, and I/O guardrails reduces SLO miss rate by about 32 percent and p99 latency by about 15 percent at under 5 percent throughput cost on a 16-GPU A100 cluster.","lead":"This paper builds a host-level controller that dynamically reconfigures MIG slices, places latency-sensitive tenants away from PCIe hot spots, and throttles I/O-heavy neighbors to cut SLO miss rate and tail latency on shared A100 clusters. It may interest anyone running latency-sensitive inference in multi-tenant GPU clouds without privileged access to the fabric.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline gains may rely on censoring MIG-reconfiguration downtime: the paper never says how requests during Relaunch(C) enter the SLO/latency metrics.","rationale":"The reader's conditional verdict is reasonable, but the weakest assumption identified by the reader (guest-visible MIG reconfiguration) is about deployability. I find a more direct threat to the measured numbers: the controller's strongest lever, dynamic MIG, requires stopping and relaunching the tenant. Table 4 and Section 5 document this pause, yet the metrics section does not specify the treatment of requests during that downtime. Without this detail, the reported miss-rate and p99 could be censored, which would invalidate the headline comparison if true. This is checkable from logs or a rerun, and it is distinct from generalizability concerns. I also considered the underspecified static-MIG baseline and the cloud-platform assumption; both matter, but the reconfiguration-window ambiguity is the most load-bearing because it directly affects the central quantitative claim. The formal model in Section 2.5 is explicitly qualitative and does not support the empirical claims, so I do not weigh it. The paper has strengths: seven repeated runs, confidence intervals, identical interference schedules, and post-change rollback validation. Those make the results plausible, but they do not resolve the censoring question. Therefore the conditional verdict stands, with an additional explicit condition to report the full trace including reconfiguration windows.","tokens_in":8579,"tokens_out":10861,"duration_ms":105921,"concrete_test":"Re-run E1 with the T1 client continuously issuing requests and logging every outcome, including requests sent while Relaunch(C) is executing; classify connection failures and timeouts as SLO misses and recompute p99 and miss-rate on the full trace. Separately measure the end-to-end time to reload the vLLM model after a MIG profile change. If the full-system miss-rate or p99 moves outside the reported confidence intervals, or if measured throughput cost exceeds the 5% budget, the headline comparison must be revised or re-scoped.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Algorithm 1 (lines 9-13) calls 'Relaunch(C)' after every isolation change, and Section 5 admits 'MIG profile changes require a brief pause of the tenant and may reload model state.' Table 4 reports MIG reconfiguration at 18±6s and fewer than 5 moves per hour, so the primary tenant is down for up to roughly 90 seconds per hour from reconfiguration alone, before any vLLM model reload. The evaluation section says it records 'the full latency distribution' and T1 throughput in RPS, but never states how requests that arrive while T1 is stopped are treated: queued, failed, timed out, or omitted from the sample. If such requests are omitted, or if the load generator is paused during reconfiguration, the reported p99 and SLO miss-rate are computed on a censored distribution, and the headline 32% miss-rate reduction and 15% p99 gain are inflated by exactly the downtime the paper acknowledges. Since dynamic MIG is the largest single contributor in the ablation (20.0→17.2 ms), this is not a peripheral issue.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a host-level controller for latency-sensitive LLM inference on shared A100 GPU clusters. The controller combines dynamic MIG reconfiguration, PCIe-aware placement, and lightweight guardrails (MPS quotas, cgroup I/O throttles), driven by per-tenant tail-latency signals and guided by a three-tiered decision policy with dwell and cool-down periods to prevent oscillation. The authors report that on a single host and a 2-node 16-GPU AWS p4d cluster, compared to static MIG and naive placement, the controller reduces SLO miss-rate by about 32%, improves p99 latency by about 15%, and incurs at most 5% throughput cost. A vLLM/OLMo 2 7B case study reports TTFT p99 improvement of about 13% at about 4% throughput cost. The paper also sketches a formal model of PCIe contention as a processor-sharing queue and states a stability theorem for throttled tenants.","tokens_in":8755,"tokens_out":4408,"duration_ms":39698,"significance":"The problem addressed is relevant: noisy-neighbor interference on shared PCIe is a recognized cause of tail latency in multi-tenant GPU inference, and a renter-deployable controller that does not require fabric privileges is a practical contribution. The empirical methodology is generally sound: 7 repeated runs with fixed seeds and 95% confidence intervals, and the reported 32% miss-rate reduction is arithmetically consistent with Table 3 (16.4% to 11.1%). The ablation study is a useful strength, showing that dynamic MIG and placement each contribute comparably. However, the formal model in Section 2.5 is explicitly qualitative, the stability theorem is only a sketch, and there are load-bearing omissions in the experimental reporting (measurement of downtime, multi-node numbers, sensitivity analysis) that currently prevent the claims from being fully supported. The approach's significance is moderate: the gains are modest but consistent, and the deployment story is clear.","major_comments":[{"comment":"The manuscript does not state how requests arriving during MIG reconfiguration or tenant pause are treated in the latency and SLO metrics. Algorithm 1 calls Relaunch(C) after every isolation change, Table 4 reports MIG reconfiguration at 18±6 s and fewer than 5 moves per hour, and Section 5 admits that 'MIG profile changes require a brief pause of the tenant and may reload model state.' This implies the primary tenant may be unavailable for up to roughly 90 s per hour. If requests that arrive during this pause are omitted from the sample, or if the load generator is paused, then the reported p99 and SLO miss-rate are computed on a censored distribution, and the headline 32% miss-rate reduction and 15% p99 improvement are inflated by exactly the downtime the paper acknowledges. The paper must specify explicitly whether such requests are queued, failed, timed out, or excluded, and if excluded, must recompute the metrics with the downtime counted as SLO violations or provide a sensitivity bound.","section":"Sections 3.2, 5, Algorithm 1"},{"comment":"The abstract claims results on 'a single host and a 2-node (16-GPU) cluster,' but no quantitative results for the multi-node case are reported in the Results section. The text states only that 'the policy shows similar improvements' on the 2-node cluster, with no table or figure giving SLO miss-rate, p99, throughput, or confidence intervals for that configuration. Table 3 appears to report only single-host ablation results. The multi-node claim in the abstract and contributions is therefore unsupported by the presented evidence. Please provide the multi-node measurements or revise the claims to match the data actually shown.","section":"Abstract and Section 4"},{"comment":"The paper lists a sensitivity analysis (E3) as one of the experiments, but the Results section contains no sensitivity results: no tables or figures for variations of tau, Y, MPS quota bounds, or I/O throttle bounds. Given that these parameters are free and are central to the controller's behavior, the absence of any reported sensitivity data weakens the robustness claim and makes it impossible to judge whether the reported improvements depend critically on hand-tuned thresholds. Either add the sensitivity results or remove the E3 experiment listing from the experimental design.","section":"Section 3.3.3 (E3) and Section 4"},{"comment":"The 'stability theorem' is presented as a formal claim, but the proof is only a sketch. The assumptions are strong (the entire PCIe fabric is a single processor-sharing server, aggregate throttles are always below capacity) and are not verified against the experimental system. However, the paper explicitly labels the model as 'qualitative guidance rather than an exact tail predictor,' so this is not a fatal flaw. Nevertheless, the theorem label overstates the result; I recommend either proving the claim rigorously under assumptions that are checked in the evaluation, or rephrasing it as a heuristic stability argument.","section":"Section 2.5.3, Claim 1"}],"minor_comments":[{"comment":"The placement score is described qualitatively ('the score penalizes...'); no concrete formula or weighting is given. For reproducibility, please define the score explicitly as an equation or pseudocode.","section":"Section 2.2.1"},{"comment":"Dwell time (256 observations) and cool-down (128 observations) are given in observation counts, while Section 2.3 discusses them as time intervals. Since the sampling interval Delta is 1–5 s, please clarify the relationship (e.g., dwell time = 256 * Delta seconds) and state the value used in the experiments.","section":"Table 1 and Section 2.3"},{"comment":"The interference script is said to 'toggle' T2 and T3, but the duty cycle, on/off durations, and phase of the toggling are not specified. Since all comparisons rely on identical interference schedules, please define the schedule precisely.","section":"Section 3.1"},{"comment":"The function Relaunch(C) is never defined in the algorithm or implementation notes. It presumably restarts the serving process and possibly reloads the model; please state exactly what Relaunch does and how long it takes, since it is directly relevant to the downtime concern.","section":"Algorithm 1"},{"comment":"The figures are referenced but are not included in the submitted text; please ensure the actual figures are present and that captions describe the data and, where appropriate, confidence intervals or error bars.","section":"Figures 3 and 4"}],"recommendation":"major_revision","confidential_remarks":"The paper is a plausible systems contribution with honest limitations, but the missing handling of reconfiguration downtime in the measurement pipeline is a genuine load-bearing issue that must be addressed empirically, not just acknowledged. The missing multi-node numbers and sensitivity results are also straightforward to fix by adding data. I would not recommend acceptance until these are resolved. The novelty claim 'First SLO-safe, multi-tenant control demo on a multi-node (16-GPU) cloud cluster without fabric privileges' is not substantiated by a comparison to prior demonstrations; this could be softened or better contextualized."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The punchline: this is a legitimate, well-scoped systems contribution — combining dynamic MIG, PCIe-aware placement, and guardrails into a host-level controller that a tenant can run with root inside a VM — and the empirical story is mostly believable. But there is a hole in the evaluation: the paper never says how requests that arrive during MIG reconfiguration's brief tenant pause are handled, and since dynamic MIG is the biggest single contributor in the ablation, that omission matters for the headline 32% SLO miss-rate reduction.\n\nWhat is actually new: MIGRator, PCIe-aware scheduling, and host-level throttles all exist in the cited literature. The contribution is the integration into one conservative controller demonstrated without fabric privileges on a 2-node/16-GPU cluster. That's a real, useful thing to show. The ablations are clean and internally consistent — the 20.0→16.5ms p99 progression matches the miss-rate numbers, 7 repeats with CIs is decent, and the vLLM/OLMo TTFT case study strengthens the story.\n\nSoft spots, in order. First, the reconfiguration downtime accounting. Algorithm 1 relaunches after every isolation change; Table 4 says MIG changes take 18±6s, <5/hour. The discussion admits a \"brief pause of the tenant.\" But Section 3.2 says they record the \"full latency distribution\" without stating whether requests during the pause are queued, failed, timed out, or dropped from the sample. If dropped, the reported p99 and miss-rate are computed on a censored distribution and the gains are overstated. This needs to be explicit. Second, no code or data released — for a systems paper with this many parameters, that hurts reproducibility, especially the placement score heuristic. Third, the baseline is underspecified: what static MIG profile, what \"naive\" placement? Fourth, the formal model in Section 2.5 is qualitative and the stability theorem is a standard PS argument; it doesn't derive the empirical gains, so it's fine to keep but shouldn't be the centerpiece.\n\nThe citation pattern looks fair — the prior work is credited, and the novelty claim is appropriately narrow. The paper does admit its small scale and synthetic workloads in Section 5. In short, the approach is credible and worth engaging, but the downtime accounting must be fixed before the headline numbers mean what they claim.\n\nI'd send this to a serious referee. For a reading group, it's a good case study in how SLO evaluation can quietly miss reconfiguration cost.","headline":"Solid integrated controller with a believable ablation, but the evaluation never accounts for MIG reconfiguration downtime, which could inflate the headline SLO gains.","tokens_in":9340,"tokens_out":2283,"would_cite":false,"duration_ms":20877,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A controller that runs inside a tenant VM cuts GPU SLO violations by 32 percent on shared A100 clusters, with no fabric privileges.","keywords":["GPU multi-tenancy","LLM serving","vLLM","TTFT","cluster scheduling","A100","SLO compliance","QoS"],"falsifier":"On a cloud platform where the guest cannot reconfigure MIG profiles or read PCIe counters (for instance, a managed GPU service that blocks nvidia-smi mig), run the same controller under the same T2/T3 interference schedule; if the SLO miss-rate remains at the static-MIG baseline, the central claim is falsified. The positive version is a reproduction on a different 16-GPU A100 cluster with the same controller and interference toggles, showing the same ≈32% miss-rate reduction at ≤5% throughput cost.","tokens_in":8343,"feed_emoji":"⚙️","tokens_out":9063,"duration_ms":71224,"temperature":0.7,"pith_summary":"The paper argues that a cloud tenant can tame noisy-neighbor interference on shared A100 GPUs without any help from the cluster scheduler or fabric privileges. Its host-level controller watches per-tenant tail latencies and PCIe activity, then takes one of three actions: resize the tenant's MIG partition for stronger hardware isolation, move the tenant to a less contended PCIe path, or throttle the offending background workload. In a controlled 2-node, 16-GPU cluster, the controller lowers the SLO miss-rate by about 32 percent and p99 latency by about 15 percent while keeping throughput loss at or below 5 percent relative to static MIG with naive placement. For LLM serving with vLLM, time-to-first-token p99 improves by roughly 10–15 percent at the same cost. The point is that predictable serving does not require privileged access to the GPU fabric.","feed_headline":"Tenant-side GPU controller cuts tail-latency misses by 32%","feed_subtitle":"It resizes GPU slices and avoids congested data paths from inside a tenant VM, at under 5% throughput cost.","key_machinery":"The central object is a three-tiered decision space: dynamic MIG reconfiguration, PCIe-aware placement, and lightweight guardrails. The argument is carried by a feedback loop with hysteresis—p99 exceeding threshold for Y consecutive windows triggers escalation, while sustained stability triggers relaxation—combined with the processor-sharing model of PCIe contention, which justifies why throttling background I/O and moving off hot PCIe paths reduce latency tails. The controller's greedy upgrade policy is guaranteed to terminate because each isolation upgrade strictly increases SM/memory resources and the configuration space is finite.","core_discovery":"The central claim is that PCIe-fabric interference, not just compute and memory contention, drives tail-latency violations in multi-tenant GPU serving, and that a tenant-visible controller can mitigate it using only three conservative levers: dynamic MIG reconfiguration, PCIe-aware placement, and lightweight guardrails (MPS quotas and cgroup I/O throttles). The paper reports that on one host and on a 2-node (16-GPU) cluster, the full controller reduces SLO miss-rate by approximately 32 percent (about 1.5×) and p99 latency by approximately 15 percent with at most 5 percent throughput cost versus static MIG and naive placement. Ablations show the MIG and placement components contribute nearly equally, with guardrails adding a smaller increment. The same controller, unchanged, improves LLM TTFT p99 by about 10–15 percent at under 5 percent throughput cost when serving OLMo 2 7B Instruct through vLLM.","pith_inferences":["If the controller is deployed on a GPU generation that exposes MIG and guest-visible PCIe counters, the same 32 percent reduction is likely to reproduce, but the paper only demonstrates it on A100; a natural next experiment is an H100 cluster with the same interference schedule.","Because the paper's interference is scripted (T2/T3 toggles), the benefit under uncoordinated production traffic may be smaller; a testable extension is to replay a real datacenter trace of PCIe and SM load.","The controller's placement heuristic is a simple penalty score; replacing it with a learned predictor could improve stability, but would add complexity and require richer signals than NVML/DCGM expose.","The paper's limitation section states that very heavy training tenants will overwhelm the controller, so the practical scope is moderate background noise rather than full training storms."],"forward_implications":["Dynamic MIG reconfiguration alone lowers p99 from 20.0 ms to 17.2 ms, PCIe-aware placement alone to 17.8 ms, and the two together to 16.5 ms, so the pair is nearly additive.","The controller's actions are gated by dwell and cool-down timers and validated before persisting, so it can run as a host-level refinement beneath cluster schedulers without causing thrash.","Because the controller needs only root access inside the tenant VM, it can be deployed on rented cloud instances without cooperation from the provider.","For LLM serving, the same controller improves TTFT p99 by roughly 10–15 percent and keeps throughput loss under 5 percent, indicating the mechanism transfers from fixed-size inference to autoregressive generation."],"supporting_citations":[{"why":"Demonstrates that MIG instances still share PCIe bandwidth, motivating the controller's PCIe-aware placement and I/O throttling levers.","marker":"[7]"},{"why":"Supplies the dynamic MIG reconfiguration principle the controller adopts for isolation upgrades.","marker":"[8]"},{"why":"Shows cluster-scale dynamic MIG exploitation, the precedent for reconfiguring MIG slices at runtime.","marker":"[3]"},{"why":"Provides background on PCIe arbitration and its role in multi-tenant GPU latency.","marker":"[4]"},{"why":"Supplies the MPS and cgroup-based host-level controls used as guardrails.","marker":"[9]"}],"fun_headline_variants":["Tenant-side GPU controller cuts SLO misses by 32%","Fabric-aware GPU slicing reduces p99 latency 15%","Dynamic MIG and placement: 32% fewer SLO violations","Host-level guardrails ease GPU tail latency by 15%","Reshaping GPU slices trims LLM TTFT p99 by 10-15%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The controller only works if the cloud lets the tenant VM dynamically change MIG profiles and read PCIe/DCGM counters from inside the guest; on platforms that hide these capabilities, none of the three action levers can fire.","fun_headline_variants_meta":{"raw":{"variants":["Tenant-side GPU controller cuts SLO misses by 32%","Fabric-aware GPU slicing reduces p99 latency 15%","Dynamic MIG and placement: 32% fewer SLO violations","Host-level guardrails ease GPU tail latency by 15%","Reshaping GPU slices trims LLM TTFT p99 by 10-15%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000176,"raw_usage":{"total_tokens":1288,"prompt_tokens":944,"completion_tokens":344,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":560,"completion_tokens_details":{"reasoning_tokens":249}},"tokens_in":560,"tokens_out":344,"duration_ms":3652,"temperature":1.0,"reasoning_tokens":249,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T16:47:03.250294+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a cloud platform where the guest cannot reconfigure MIG profiles or read PCIe counters (for instance, a managed GPU service that blocks nvidia-smi mig), run the same controller under the same T2/T3 interference schedule; if the SLO miss-rate remains at the static-MIG baseline, the central claim is falsified. The positive version is a reproduction on a different 16-GPU A100 cluster with the same controller and interference toggles, showing the same ≈32% miss-rate reduction at ≤5% throughput cost.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the MPS and cgroup-based host-level controls used as guardrails."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Demonstrates that MIG instances still share PCIe bandwidth, motivating the controller's PCIe-aware placement and I/O throttling levers."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides background on PCIe arbitration and its role in multi-tenant GPU latency."}],"review_version":2}