{"id":"aa1d4163-20f0-4562-a277-c1fdf22739c5","arxiv_id":"2512.16056","paper_version":2,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"MMA routes host-GPU transfers over multiple available paths to deliver 4.62x higher peak bandwidth and lower latencies in LLM serving without hardware or driver changes.","lead":"This paper introduces Multipath Memory Access (MMA), a software system that splits host-to-GPU data transfers across multiple PCIe and interconnect paths in multi-GPU servers. Smart generalists might care because faster data movement can reduce startup and memory-fetch delays when running large AI models on existing hardware.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Relay-path coordination overhead via Dummy Task and lightweight sync not isolated from bandwidth gains","rationale":"Reader's weakest assumption directly identifies the same load-bearing point. Because the original review had only the abstract, the full text would need explicit ablation data on coordination overhead to move beyond UNVERDICTED; absent that isolation, the empirical numbers remain conditional on unverified low-overhead behavior.","tokens_in":1804,"tokens_out":385,"duration_ms":20317,"concrete_test":"Instrument a single 4 GB host-to-GPU transfer: run native cudaMemcpyAsync and MMA side-by-side on the same 8-GPU H20 server, record (a) achieved bandwidth, (b) wall-clock time from launch to completion including all internal sync, and (c) GPU/CPU utilization during the transfer. If MMA's effective bandwidth falls more than 15 % below the sum of measured path capacities or adds >5 ms latency beyond native, the multipath claim does not hold at the reported scale.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The headline 4.62x bandwidth improvement and 1.14-2.38x TTFT reductions rest on the assumption that splitting transfers across direct PCIe plus relay paths (peer-GPU PCIe + high-bandwidth interconnects) incurs negligible coordination cost. The described mechanisms—dependency-preserving Dummy Task, distributed micro-transfer completion sync, and queue backpressure—must add near-zero latency and CPU/GPU resource usage while preserving full CUDA stream semantics. No evidence is given that these costs were measured in isolation (e.g., single-stream microbenchmarks subtracting native copy time from multipath time) or under concurrent LLM-serving streams where multiple KV-cache fetches and weight loads overlap. If sync or dummy-task overhead exceeds a few percent, the net gain collapses and the claimed latency reductions become unreliable.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper introduces MultiPath Memory Access (MMA), a software-defined system that splits host-GPU transfers across direct PCIe paths and relay paths through peer GPUs and high-bandwidth interconnects. It uses a dependency-preserving Dummy Task, lightweight distributed synchronization for micro-transfer completion, and queue backpressure to preserve CUDA stream semantics without hardware, driver, or application changes. On an 8-GPU NVIDIA H20 server, MMA reports 245 GB/s peak host-to-GPU bandwidth (4.62x over native CUDA copies), 1.14-2.38x TTFT reduction for KV cache fetching, and 1.12-2.48x reduction in model wake-up/switching latency.","tokens_in":1942,"tokens_out":544,"duration_ms":21675,"significance":"If the performance claims hold after overhead isolation, MMA would meaningfully improve LLM serving efficiency by utilizing otherwise-idle intra-server I/O capacity for latency-critical paths such as weight loading and KV cache movement, without requiring new hardware.","major_comments":[{"comment":"§5 (Evaluation): the headline 4.62x bandwidth and 1.14-2.38x TTFT claims rest on the unverified assumption that Dummy Task insertion, distributed micro-transfer synchronization, and queue backpressure add negligible latency and resource usage. No single-stream microbenchmarks that subtract native copy time from multipath time, nor measurements under concurrent LLM-serving streams, are reported to isolate these costs.","section":"§5"},{"comment":"§4 (Design) and §5: the claim that queue backpressure routes traffic correctly without explicit link-state feedback while preserving full CUDA stream semantics is load-bearing for correctness under overlapping KV-cache and weight transfers, yet no evaluation of stream ordering or application-visible correctness under realistic multi-stream LLM workloads is provided.","section":"§4 and §5"}],"minor_comments":[{"comment":"Abstract and §5: results lack error bars, precise workload descriptions (e.g., model sizes, batch sizes, concurrency levels), and ablation tables separating bandwidth gains from coordination overhead.","section":"Abstract and §5"},{"comment":"Figure clarity: diagrams of relay-path micro-transfers and Dummy Task insertion would benefit from explicit timing annotations to illustrate dependency preservation.","section":null}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a good fit for cs.DC; the evaluation section would be substantially strengthened by adding the requested overhead-isolation experiments, which appear feasible within the existing experimental setup."},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the constructive feedback emphasizing evaluation rigor. We address each major comment below and will revise the manuscript to include the requested microbenchmarks and correctness evaluations.","responses":[{"response":"We agree that dedicated isolation of overheads from Dummy Task insertion, micro-transfer synchronization, and queue backpressure would strengthen the claims. The reported 245 GB/s bandwidth and TTFT/latency speedups are end-to-end measurements that already incorporate any such costs. To directly address the concern, we will add single-stream microbenchmarks subtracting native copy time from MMA time and concurrent-stream measurements under realistic LLM workloads to quantify latency and resource overhead. These will appear in the revised §5.","revision_made":"yes","referee_comment":"[§5] §5 (Evaluation): the headline 4.62x bandwidth and 1.14-2.38x TTFT claims rest on the unverified assumption that Dummy Task insertion, distributed micro-transfer synchronization, and queue backpressure add negligible latency and resource usage. No single-stream microbenchmarks that subtract native copy time from multipath time, nor measurements under concurrent LLM-serving streams, are reported to isolate these costs."},{"response":"Queue backpressure is intended to preserve CUDA stream semantics via dependency-preserving Dummy Tasks and lightweight synchronization without needing explicit link-state feedback. While the current evaluation does not include a dedicated stream-ordering test, the TTFT and wake-up latency results were obtained from realistic multi-stream LLM serving workloads involving overlapping KV-cache and weight transfers; any violation of ordering would have produced incorrect results or crashes. We will add explicit stream-ordering and application-correctness experiments under multi-stream workloads to the revised §5.","revision_made":"yes","referee_comment":"[§4 and §5] §4 (Design) and §5: the claim that queue backpressure routes traffic correctly without explicit link-state feedback while preserving full CUDA stream semantics is load-bearing for correctness under overlapping KV-cache and weight transfers, yet no evaluation of stream ordering or application-visible correctness under realistic multi-stream LLM workloads is provided."}],"tokens_in":1436,"tokens_out":455,"duration_ms":20333,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The paper's core contribution is a software layer that splits host-to-GPU transfers across direct PCIe and relay routes through peer GPUs and their interconnects, all while keeping CUDA stream semantics. They use a dummy task to preserve dependencies, a lightweight distributed sync for micro-transfer completions, and queue backpressure for routing decisions. On an 8-GPU H20 server this produces 245 GB/s peak bandwidth (4.62x over native copies) and cuts TTFT and model switch times by 1.14-2.48x under the workloads they tested. That is a concrete systems win for LLM serving where weight loads and KV cache moves are frequent bottlenecks, and it does so without driver or application changes. The approach is new in its specific combination of dependency preservation and feedback-free routing inside a single server. The measurements come from actual hardware rather than models, which gives the claims some grounding. The main soft spot is the coordination overhead. The dummy-task and sync mechanisms must stay near zero cost, yet the evaluation does not isolate those costs with single-stream microbenchmarks or show how they behave when multiple concurrent streams overlap. Without error bars, workload details, or an ablation that subtracts the added latency from the bandwidth gain, it is hard to know how much headroom remains under heavier serving loads. The central claim still looks plausible from the numbers given. This paper is for systems builders working on LLM inference stacks who already have multi-GPU nodes and want to improve I/O utilization. A reader focused on serving latency would find the design and the reported speedups useful. I would send it to peer review; the empirical results are strong enough to justify referee time even if the overhead analysis needs expansion.","headline":"MMA shows a workable way to pull extra host-GPU bandwidth from existing multi-GPU servers by routing over relay paths, with real measured gains, but the overhead accounting still needs tighter checks.","tokens_in":2470,"tokens_out":426,"would_cite":true,"duration_ms":29728,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":{"model":"grok-4.3","evidence":[{"relation":"unclear","rs_module":"IndisputableMonolith/Cost/FunctionalEquation.lean","rs_theorem":"washburn_uniqueness_aczel","paper_passage":"MMA introduces a novel dummy task that retrieves control from asynchronous transfer tasks, along with a synchronization mechanism to maintain the original dependencies... congestion-aware routing strategy that achieves intra-server traffic load balancing even in the absence of path-aware information."},{"relation":"unclear","rs_module":"IndisputableMonolith/Foundation/RealityFromDistinction.lean","rs_theorem":"reality_from_one_distinction","paper_passage":"On an 8-GPU NVIDIA H20 server, MMA achieves 245 GB/s peak host-to-GPU bandwidth, a 4.62x improvement over native CUDA copies"}],"headline":"MMA multipath CUDA relay engine has no structural overlap with RS forcing chain","alignment":"orthogonal","rationale":"The paper's core machinery (Dummy Task interception, Sync Engine with spin kernels, Outstanding-queue passive load balancing over PCIe+NVLink relays, dual-pipeline relay) is a practical systems hack for host-GPU DMA in LLM serving. It operates entirely within CUDA runtime semantics and intra-server interconnect topology. RS derives J-cost, φ-ladder constants, 8-tick periodicity, D=3, and spacetime from a single distinction (reality_from_one_distinction, AbsoluteFloorClosure, Cost/FunctionalEquation, AlexanderDuality). No shared primitives, cost functions, ratio symmetry, or parameter-free derivations appear; the domains are disjoint.","tokens_in":54140,"confidence":"high","tokens_out":361,"duration_ms":9558,"cache_read_input_tokens":38528,"cache_creation_input_tokens":0},"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Multipath Memory Access routes host-GPU copies over unused server links to raise bandwidth 4.6x.","keywords":["multipath memory access","host-GPU bandwidth","LLM serving","KV cache offload","multi-GPU server","CUDA optimization","data transfer"],"falsifier":"Measure the sustained host-to-GPU copy bandwidth on the 8-GPU server with a single large transfer both with and without MMA enabled; if MMA does not exceed the native single-path limit by a large margin, the claim fails.","tokens_in":2695,"feed_emoji":"⚡","tokens_out":730,"duration_ms":38100,"temperature":0.7,"pith_summary":"Host-GPU data transfers in LLM services are stuck using only the direct PCIe link to one GPU, leaving other server paths idle. MMA spreads each transfer across multiple paths by relaying data through peer GPUs and their interconnects. The system keeps all existing CUDA code working unchanged by using a dummy task to preserve stream dependencies and a simple backpressure mechanism to coordinate the paths. This raises peak bandwidth from the native limit to 245 GB/s on an 8-GPU H20 server and cuts the time for fetching KV caches and switching models.","feed_headline":"Multipath routing lifts host-GPU bandwidth 4.6x","feed_subtitle":"MMA uses peer-GPU relays to unlock unused PCIe and interconnect capacity in LLM servers.","key_machinery":"Multipath Memory Access (MMA), which splits transfers into micro-operations routed over direct PCIe links and relay paths through peer GPUs connected by high-bandwidth interconnects.","core_discovery":"MMA expands a single host-GPU copy across available direct and relay paths without hardware, driver, or application changes. It preserves CUDA stream semantics with a dependency-preserving Dummy Task, coordinates distributed micro-transfer completion through a lightweight synchronization mechanism, and uses queue backpressure to route traffic without explicit link-state feedback. On an 8-GPU NVIDIA H20 server, MMA achieves 245 GB/s peak host-to-GPU bandwidth, a 4.62x improvement over native CUDA copies, and reduces TTFT for KV cache fetching by 1.14-2.38x and model wake-up/switching latency by 1.12-2.48x.","pith_inferences":["The same multipath idea could apply to other multi-device setups where data must cross host-device boundaries.","If interconnect speeds increase in future hardware, the relative gain from MMA may grow because more relay capacity would be available.","Workloads with very small transfers might see less benefit if the overhead of splitting and synchronizing exceeds the path gains.","Testing on servers with different GPU counts or interconnect topologies would show how well the routing scales."],"forward_implications":["Effective host-GPU bandwidth increases without buying new hardware or changing drivers.","KV cache offload and fetch operations complete faster, lowering time-to-first-token in LLM inference.","Model loading and switching between different LLMs happens with less delay in shared servers.","Existing LLM serving frameworks can adopt the gains immediately since no code changes are required.","Server I/O capacity that was previously unused becomes available for data movement."],"fun_headline_variants":["MMA achieves 245 GB/s host-GPU bandwidth","4.6x host-GPU bandwidth with multipath MMA","MMA reduces KV cache fetch time 1.14-2.38x","Peer-GPU paths enable multipath data transfer"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The extra relay paths can be used without adding enough overhead to cancel out the bandwidth gains while still obeying all CUDA ordering rules.","fun_headline_variants_meta":{"raw":{"variants":["MMA achieves 245 GB/s host-GPU bandwidth","4.6x host-GPU bandwidth with multipath MMA","MMA reduces KV cache fetch time 1.14-2.38x","Peer-GPU paths enable multipath data transfer"]},"model":"grok-4.3","cost_usd":0.00752,"raw_usage":{"total_tokens":3494,"prompt_tokens":757,"num_sources_used":0,"completion_tokens":67,"cost_in_usd_ticks":75199500,"prompt_tokens_details":{"text_tokens":757,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2670,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":757,"tokens_out":67,"duration_ms":23801,"temperature":1.0,"reasoning_tokens":2670,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-05-16T21:54:28.884047+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Measure the sustained host-to-GPU copy bandwidth on the 8-GPU server with a single large transfer both with and without MMA enabled; if MMA does not exceed the native single-path limit by a large margin, the claim fails.","supporting_citations":[],"review_version":1}