{"id":"e45df406-1679-4c6d-bc41-95c10a56b281","arxiv_id":"2607.11136","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"low","formal_verification":"none","parameter_count":3,"one_line_summary":"Trace-guided fine-grained memory control and offline joint planning raise diffusion serving SLO attainment by up to 3.7× while cutting configuration search from hours to minutes.","lead":"Xema is a diffusion serving system that uses offline memory traces to apply GPU memory mitigation only in short pressure intervals, plus a static tensor layout and an auto-planner for parallelism and batching. It can make high-resolution image and long-video requests fit limited GPUs with much higher SLO attainment than coarse offloading.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"No significant objection identified beyond the reader's already-flagged template-static assumption; that assumption is the right load-bearing hinge and is not newly undermined by the text.","rationale":"The reader's weakest assumption is exactly the condition that must hold for offline traces, static layout, and plan-table lookup to be reliable at runtime. The manuscript does not hide this: it argues template-static execution and localized pressure (§3.3, Fig. 2), then builds mitigation and planning on that (§5–7). Reported results (3.7× on Flux.2 at 8 req/min, extreme templates fitting one A6000, planning 197 s vs 6.3 h) cohere with the mechanisms and ablations; I do not see a more load-bearing internal gap (e.g., mis-specified goodput, circular SLO, or contradiction between memory analyzer and runtime). Evaluation limits justify CONDITIONAL rather than unconditional ACCEPT, which the reader already chose with high confidence. Stress-test therefore leaves the verdict unchanged and agrees with the reader on the hinge.","tokens_in":21408,"tokens_out":620,"duration_ms":7496,"concrete_test":"On CogVideoX-5B 1080p×289f and Flux.2 2048², run the offline shape-only trace and static layout, then execute the same plan on real prompts (varied text, CFG, seeds) while logging reserved vs active peaks and any OOM or plan-table miss; if reserved peak exceeds the offline plan+5% margin by >10% or SLO attainment collapses relative to Fig. 9/11, the template-static hinge fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (up to 3.7× SLO attainment and planning cut from 6.3 h to 197 s via interval-local mitigation plus joint planning) rests on the template-static contract the reader already names: for fixed (model, H, W, T) and step count, a shape-only run yields operator sequence, tensor shapes, and lifetimes that match runtime for managed tensors (§3.3; §6.1; Algorithm 1). The paper states data-independent control flow and treats dynamic memory (CUDA/NCCL/contexts) as a small ~5% margin. I do not find a stronger internal inconsistency: ablations (Fig. 14), extreme-template feasibility (Fig. 11), allocator traces (Figs. 12–13), and planning breakdown (Table 1) are mutually consistent with that premise. Residual risk is empirical scope (single A6000 node, synthetic mixes, SLO relative to the shared candidate space, no public code), which the reader already used for CONDITIONAL—not a new load-bearing flaw in the argument.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"Xema is a diffusion serving system that treats request templates (model, resolution, frame count) as template-static so that offline shape-only runs can produce memory traces and tensor lifetimes. It applies memory mitigation only inside short pressure intervals and only at the intensity needed to fit a GPU budget, builds a static 2D best-fit layout for graph-visible tensors to cut fragmentation, and uses an offline planner that jointly chooses parallelism, batch size, and fine-grained memory controls under memory and SLO constraints, storing the result in a plan table for online lookup. Implemented on Diffusers/xDiT and evaluated on Flux.2, CogVideoX-5B, and LTX-2 against vLLM-Omni, globally extended primitives, and a greedy heuristic, the paper reports up to 3.7× higher SLO attainment and planning cost reduced from 6.3 hours to 197 seconds versus grid search.","tokens_in":21747,"tokens_out":965,"duration_ms":9939,"significance":"If the results hold, the paper is a solid systems contribution for production diffusion serving: it shows that coarse global offloading/slicing is often the wrong granularity, that localized mitigation plus static layout can make extreme image/video templates feasible on 48 GB GPUs, and that joint planning of SP/DP, batch size, and interval-local controls measurably improves goodput. Strengths include multi-model end-to-end SLO curves (rate, GPU count, SLO scale, skewed mix), extreme single-GPU feasibility, allocator vs. expandable_segments comparison, planning-cost breakdown, and configuration ablations. The work is empirical rather than formal, but the evaluation design is appropriate for the claim and the planning-cost reduction is concrete.","major_comments":[{"comment":"The central claim rests on the template-static contract (§3.3, §6.1): for fixed (model, H, W, T) and denoising steps, a shape-only run yields operator sequence, tensor shapes, and allocation/free events that match runtime for managed tensors, with dynamic memory covered by a fixed ~5% margin. The paper asserts data-independent control flow but does not stress-test content-dependent paths, adaptive step counts, or runtime buffers that could break offline traces and the static allocator. A short sensitivity study or explicit failure modes would make the load-bearing assumption falsifiable rather than assumed.","section":null},{"comment":"§9.1 defines the base SLO as 2.5× the latency of the best feasible strategy in the shared candidate space. Because Xema expands that space with fine-grained controls, the relative SLO can favor configurations that only Xema can reach and makes the 3.7× attainment figure harder to interpret against systems that cannot serve the same templates. Reporting absolute latency/goodput or an external fixed-latency SLO would strengthen the comparison.","section":null},{"comment":"Evaluation is limited to a single 8×A6000 PCIe node with synthetic Poisson mixes (§9.1). Multi-node interconnects, production traffic, and other GPU generations are not shown; the planner’s communication/latency tradeoffs (SP, HSDP) may change under different topologies. This does not invalidate the single-node results but bounds how far the goodput claims can be generalized without additional evidence.","section":null}],"minor_comments":[{"comment":"Figure 1 caption and bars are useful but the exact configuration strings (e.g., “offload 3 blocks + fusion”) are dense; a short legend or table of the winning plans would help.","section":null},{"comment":"Algorithm 1 is clear; stating the complexity or typical |T| for Flux.2/CogVideoX would help readers judge offline layout cost.","section":null},{"comment":"§5.1 objective minimizes sum of latency overheads under per-segment memory constraints; clarifying whether L_i is measured end-to-end or per-segment and how overlap is credited would reduce ambiguity.","section":null},{"comment":"Related work (§10) is appropriate; a one-sentence contrast with training-oriented rematerialization/swapping systems on why diffusion’s short pressure intervals change the design would sharpen the positioning.","section":null},{"comment":"Minor polish: “V AE” spacing in §2.2; ensure consistent use of “goodput” vs. “SLO attainment” when citing the 3.7× claim.","section":null}],"recommendation":"minor_revision","confidential_remarks":"Fit for a systems venue is good. The main residual risk is empirical scope and the untested edge of the template-static assumption, not an internal contradiction. I would not block on code release, but artifact availability would raise confidence given the planner and allocator claims."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The useful takeaway is that Xema turns the template-static nature of diffusion into a practical serving advantage: offline memory traces, mitigation only on short pressure intervals (and only as much as needed), a lifetime-aware static layout, and a joint planner over SP/DP, batch size, and those controls. That combination is what is new; the individual knobs (offload, HSDP, chunking, packing) are not.\n\nWhat they do well is the systems craft. The problem framing is clear—weights vs DiT vs VAE peaks, reserved vs active memory, coarse global controls that overpay latency. Algorithm 1 and the static/dynamic split make offline reasoning match runtime for managed tensors; Figs. 12–13 show reserved memory tracking active peaks and avoiding OOM that pure caching hits. End-to-end results on Flux.2, CogVideoX-5B, and LTX-2 are multi-axis (rate, GPU count, SLO scale, skew), with sensible baselines (vLLM-Omni, global extended lib, greedy heuristic) and ablations that show all three planning dimensions matter. Extreme single-GPU feasibility and the 6.3 h → 197 s planning cut are concrete. Math is light optimization/pruning, not load-bearing theory; citations cover serving, acceleration, and memory work without obvious gaps.\n\nSoft spots, in proportion: evaluation is one A6000 node, synthetic Poisson/skew mixes, and SLOs defined as 2.5× best feasible latency in the shared candidate space—so the 3.7× is real under that protocol, not a universal production number. No public code. The hinge the reader names is correct: shape-only traces and a ~5% dynamic margin assume data-independent control flow and small runtime buffers; if that breaks, plans and the static allocator mis-predict. The stress-test does not find a stronger internal flaw; ablations and traces are consistent with the premise.\n\nThis is for people building or operating diffusion/video serving fleets and for ML-systems readers who care about memory-aware auto-config. It deserves a serious referee. I would engage with it and cite the interval-local + static-layout idea when writing about generative serving memory.","headline":"Solid systems paper: interval-local mitigation plus lifetime layout and joint planning deliver real SLO and planning-cost gains on production diffusion pipelines; main risks are scope and the template-static hinge, not internal inconsistency.","tokens_in":22320,"tokens_out":569,"would_cite":true,"duration_ms":7312,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Diffusion serving can fit high-resolution images and long videos on limited GPUs by cutting memory only in short predicted pressure windows and auto-picking the plan that maximizes goodput.","keywords":["diffusion serving","GPU memory management","trace-guided mitigation","static memory layout","auto-configuration","goodput","sequence parallelism","SLO attainment"],"falsifier":"Run the same request templates with content that triggers data-dependent control flow, dynamic shapes, or large unplanned runtime buffers; if reserved peaks then diverge from the offline plan or the static allocator OOMs while active memory still fits, the central predictability claim fails.","tokens_in":22329,"feed_emoji":"🖼️","tokens_out":1010,"duration_ms":13433,"temperature":0.7,"pith_summary":"Production image and video diffusion often fails on common GPUs because peak memory from weights, DiT activations, VAE activations, and allocator fragmentation exceeds capacity even at batch size one. Coarse global tricks such as full weight offloading, sharding, or VAE slicing fix the fit but add large latency that wrecks service-level objectives. Xema starts from a simple fact: for a fixed request template the operator sequence and tensor lifetimes are known offline, and near-peak memory appears in only a few short intervals. It builds an offline memory trace, mitigates only inside those intervals and only by the amount needed to stay under budget, and lays out predictable tensors statically so reserved memory tracks live tensors. An offline planner then jointly chooses parallelism, batch size, and these fine-grained controls, stores the best plan, and the online runtime just looks it up. On Flux.2, CogVideoX-5B, and LTX-2 the result is up to 3.7× higher SLO attainment and planning that finishes in minutes instead of hours.","feed_headline":"Cut GPU memory only where diffusion peaks, gain 3.7× SLO hits","feed_subtitle":"Offline traces plus a static layout let servers fit big image and video jobs without paying full offload cost.","key_machinery":"Trace-guided fine-grained memory mitigation plus a static lifetime-based layout: for each request template an offline shape-only run produces tensor lifetimes and a reserved-memory trace; short intervals that exceed the budget receive the minimum intensity of offloading, HSDP, chunking, fusion, or temporary serialization; non-overlapping tensors share address ranges so reserved memory closely matches active memory and runtime matches the offline plan.","core_discovery":"Because diffusion inference for a fixed request template is template-static and its memory pressure is localized, a system can derive an offline memory trace, apply mitigation only where and by the amount needed to fit a GPU budget, pin predictable tensors into a static layout that removes fragmentation, and then automatically select the highest-goodput combination of parallelism, concurrency, and memory control under memory and SLO constraints—yielding large gains in feasible request range and SLO attainment over coarse global memory controls and grid search.","pith_inferences":["The same lifetime-trace idea could extend to other template-static generative pipelines (e.g., multi-stage image editing or 3D diffusion) where peaks are brief and shapes are known offline.","If future models introduce heavy data-dependent branching, hybrid plans that re-verify only the dynamic regions while keeping static layout for the rest may still preserve most of the gain.","Operators may start publishing request-template catalogs with precomputed plans rather than only model weights, turning memory fit into a first-class deployment artifact."],"forward_implications":["High-resolution image and long-video templates that currently OOM under global offloading or VAE slicing become serviceable on the same GPUs.","Serving systems can replace multi-hour grid search with a plan table filled in minutes per template and hardware budget.","Goodput under tight SLOs rises because mitigation cost is paid only in the few percent of intervals that actually threaten the budget.","Allocator fragmentation ceases to be a silent source of OOM: reserved memory tracks active memory closely enough that offline reasoning is trustworthy at runtime.","The same planner can retarget different GPU counts and memory budgets without manual retuning of SP, DP, batch size, and offload depth."],"fun_headline_variants":["Trace-guided cuts only at diffusion peaks deliver 3.7× SLO attainment","Static tensor layouts plus peak-only mitigation fit bigger image and video jobs","Offline memory traces auto-plan parallelism to hit SLOs under GPU budgets","Localized mitigation and plan tables cut diffusion serving search from hours to seconds","Template-static lifetimes let Xema shave memory only where Flux and video pressure spikes"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"For a fixed model, resolution, frame count, and denoising-step count, the full operator sequence, tensor shapes, and allocation and free events are fixed by architecture and do not depend on the actual prompt content, so a shape-only offline run predicts the real reserved-memory peaks.","fun_headline_variants_meta":{"raw":{"variants":["Trace-guided cuts only at diffusion peaks deliver 3.7× SLO attainment","Static tensor layouts plus peak-only mitigation fit bigger image and video jobs","Offline memory traces auto-plan parallelism to hit SLOs under GPU budgets","Localized mitigation and plan tables cut diffusion serving search from hours to seconds","Template-static lifetimes let Xema shave memory only where Flux and video pressure spikes"]},"model":"grok-4.5","effort":"low","cost_usd":0.005392,"raw_usage":{"total_tokens":1448,"prompt_tokens":824,"num_sources_used":0,"completion_tokens":102,"cost_in_usd_ticks":53920000,"prompt_tokens_details":{"text_tokens":824,"audio_tokens":0,"image_tokens":0,"cached_tokens":0},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":522,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":824,"tokens_out":102,"duration_ms":7238,"temperature":1.0,"reasoning_tokens":522,"cache_read_input_tokens":0,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-14T06:48:06.258679+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Run the same request templates with content that triggers data-dependent control flow, dynamic shapes, or large unplanned runtime buffers; if reserved peaks then diverge from the offline plan or the static allocator OOMs while active memory still fits, the central predictability claim fails.","supporting_citations":[],"review_version":1}