{"id":"4d92dad6-2afa-45b3-a7d9-08c22ac262f9","arxiv_id":"2608.12114","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"File-backed weight adoption, which maps checkpoint files into shared memory and imports them into PyTorch or MLX as no-copy GPU tensors, removes the 'ingestion tax' on unified-memory systems at no measured throughput cost.","lead":"This paper shows that large AI models can run faster on computers where the CPU and GPU share memory by letting the GPU read the model file directly from the operating system's file cache instead of copying it into a framework-owned buffer. The authors call the avoided copying the 'ingestion tax' and provide a recipe plus measurements showing equal or better speed and much lower memory use on several machines.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"No significant objection identified: the page-cache residency precondition is explicit, measured, and gated; the central no-rate-cost claim is correctly scoped to that regime.","rationale":"I reviewed the central claim as conditional: under C1-C3 and page-cache residency, file-backed adoption reaches resident-class bandwidth and buys shared, reclaimable ownership. That claim is supported by the paired mapped-resident campaign (Section 3.2, [-0.66%, +0.48%]), the matched Qwen2.5-72B runs (Table 6), the bitwise logits, the cross-architecture controls (APU, GH200, discrete GPU), and the explicit failure cases (Section 6.7). The factorial decomposition is careful, the measurement traps are documented and corrected, and the appendices supply a consistent measurement chain. I did not find an internal inconsistency or a hidden mathematical assumption. The reader's weakest assumption is the page-cache residency precondition; I agree it is the most sensitive condition, but the paper states it in Section 10, checks it in Algorithm 2, and treats failure as a regime boundary, so it does not undermine the conditional claim. The only residual concern is dynamic: residency is checked at deployment time rather than enforced, so later memory pressure can invalidate the precondition after adoption. That limits robustness in shared environments but is a stated boundary, not a flaw in the argument. The reason for a CONDITIONAL rather than ACCEPT verdict remains the missing artifact and proprietary headline weights, not a correctness defect in the paper's reasoning.","tokens_in":27449,"tokens_out":11769,"duration_ms":115364,"concrete_test":"Run the public adoption Qwen2.5-72B workload while a background process gradually allocates memory after the Algorithm 2 check; sample mincore and tok/s per token. If tok/s remains at resident-class values while all sampled pages stay resident, the deployment rule is robust; if throughput collapses only when mincore reports eviction, the residency precondition is confirmed as the operative boundary; if throughput collapses while residency is 100%, an additional undiagnosed dependency exists. This check distinguishes an explicit scope boundary from unsupported robustness.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most delicate condition in the argument is the page-cache residency precondition (Section 10; Algorithm 2 lines 2-9). It is genuinely load-bearing: if the active mapped set is evicted during decode, the adopted path falls to storage speed and the in-memory parity comparison no longer applies. The paper does not hide this: Section 10 states it as a precondition, Algorithm 2 checks it before deployment, Section 6.7 shows a 2.74 GB working set failing with 0% of the mapping cached, and Section 6.6 quantifies the eviction boundary. The headline numbers are all measured with residency confirmed, so the central claim is conditional rather than overclaimed. The residual soft spot is dynamic: the algorithm samples residency at deployment time and does not pin or continuously monitor it, so a later allocator can evict the clean mapped pages and silently drop throughput to storage-bound levels. That is a robustness limitation for multi-tenant settings, not an internal inconsistency or a false measurement. It does not change the verdict.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper identifies an 'ingestion tax' on unified and coherent-memory systems: tensor frameworks copy page-cache-resident checkpoint bytes into framework-owned allocations even though the original file pages are already GPU-readable. The authors propose file-backed weight adoption, in which a framework-independent producer maps each tensor with MAP_SHARED, wraps the pages as a no-copy Metal buffer, and exports a DLPack capsule that PyTorch or MLX imports as ordinary storage. They establish a three-condition execution contract (adopt the mapped pages, keep activations accelerator-resident, order on the GPU) via a randomized same-harness factorial, and show that under the contract adoption reaches resident-class bandwidth: 516 GB/s where default constructors reach 53–82 GB/s, with a mapped-minus-resident difference bounded by [-0.66%, +0.48%]. End-to-end results include Qwen2.5-72B at 7.14 tok/s versus 0.94 tok/s for per-use ingestion and 7.23 tok/s for a load-once resident control, a 6.4x first-token speedup for a 65 GB checkpoint, a 1.21x improvement in llama.cpp on an AMD APU, a tie with overlapped streaming on a capacity-exceeding GH200, and a 39x loss on a discrete PCIe GPU. The deployment rule (Algorithm 2) chooses among map, resident, and stream-copy endpoints based on full-working-set probes. The paper explicitly states that the central claim is conditional on the active mapped set remaining in the page cache, and it measures and gates this precondition.","tokens_in":27575,"tokens_out":20153,"duration_ms":157566,"significance":"If the results hold, this is a practically important and unusually well-measured mechanism for eliminating a real inefficiency in local LLM inference on integrated and coherent-memory systems. The empirical methodology is disciplined: a randomized factorial with per-process arm rotation and twelve-pass warm-up discard, an instrument chain covering residency probes, mutation visibility, checksums, and output oracles, paired bootstrap intervals, disclosed measurement traps, and a prospective held-out machine that tests the deployment rule before the end-to-end runs. The central claim is conditional on page-cache residency, but that condition is explicit, measured, and gated, so the paper does not overclaim. The cross-architecture results (APU, GH200, PCIe) ground the topology-dependent placement rule, and the deployment heuristic yields falsifiable predictions. The paper also contributes a clear decomposition of the three execution requirements and shows that ordering costs are tied to host visibility rather than queue ownership, which is a useful systems insight.","major_comments":[],"minor_comments":[{"comment":"The text states that 'the storage change contributes 1.6× on identical files', but the reported TTFT values (rebind 6.97 s vs. copy control 8.42 s) give a ratio of about 1.21×, not 1.6×. Please correct the number or clarify what comparison yields 1.6×.","section":"§6.4"},{"comment":"The residency check in Algorithm 2 is performed only at deployment time. The paper does not specify a runtime mechanism to detect later eviction of the mapped set; given the RQ3 results showing that hot competitors can evict mapped pages, a deployment on a shared or multi-tenant machine could silently fall back to storage-bound throughput. Consider adding a runtime guard or explicitly stating that the rule assumes no subsequent memory pressure.","section":"§8.1 (Algorithm 2)"},{"comment":"The row 'production spine, live 508–524' is labeled 'live', but Section 6.1 and Table 4 report a live spine stage of 0.35 s, which is about 162 GB/s for the 52.8 GiB spine. Please clarify whether this row refers to the replay harness rather than the live system, or explain the discrepancy.","section":"Table 13"},{"comment":"The abstract and Finding (RQ2) say a 65 GB checkpoint reaches its first token '6.4× sooner (1.6× from storage alone under identical files)'. As noted above, the 1.6× figure appears inconsistent with the reported copy-control ratio; this needs correction because the decomposition is a quantitative claim in a headline finding.","section":"Abstract / §6.4"},{"comment":"The statement 'sweeping the band θ from 2% to 20% changes no verdict in Table 9; only below the GH200 pair's own 1.4% gap does that tie resolve to stream-copy' is slightly misleading: because the sweep starts at 2%, it never tests the region below 1.4% where the GH200 verdict would change. Please report the sensitivity boundary explicitly or rephrase the 'not load-bearing' claim.","section":"§8.1"},{"comment":"The paper repeatedly calls the adoption path 'public' and states that the mechanism is framework-independent, but it does not provide a repository URL or an artifact-availability statement. Please state where the 226-line producer extension and the llama.cpp patch will be made available for reproducibility.","section":"§5.1 / §7.1"}],"recommendation":"minor_revision","confidential_remarks":"The paper is well-executed and the central claim is appropriately conditional on page-cache residency, which the authors explicitly state and gate. The main residual concern is that the deployment rule lacks a runtime eviction guard, making it somewhat fragile in multi-tenant settings; this is a robustness limitation rather than an internal inconsistency. The absence of a code repository may matter for the venue's reproducibility requirements. The empirical methodology is a clear strength, especially the prospective held-out prediction, which is rare in systems papers."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a careful empirical systems study: the central claim—mapped checkpoint pages can be adopted as ordinary framework storage and read at resident-class bandwidth under a three-condition execution contract—is measured with unusual discipline and scoped honestly. The components are prior art, but the producer-plus-contract-plus-deployment-rule package is new, and the evidence is strong.\n\nThe causal decomposition is the real contribution. They separate adoption (C1), activation residency (C2), and GPU-side ordering (C3), then price each with a randomized same-harness factorial. The finding that a host-visible drain costs an order of magnitude more than an on-GPU event, and that ordering only matters when activations are resident, is the kind of result that changes how loaders are built. The cross-platform work is also credible: the same mechanism speeds up llama.cpp on an APU, ties overlapped streaming on GH200, and loses 39x on PCIe, so they don't overclaim. The held-out M3 sessions are genuine predictions, and the appendices are full of disclosed measurement traps that most systems papers skip. The citation pattern is fair; they explicitly disclaim novelty for the components and engage with the closest prior MLX prototype.\n\nSoft spots, in proportion. First, no code ships: the producer is called 'public' but there's no artifact, commit hash, or reproduction script. That alone prevents a straight accept. Second, the page-cache residency precondition is load-bearing—if the OS evicts the mapped pages mid-decode, the path falls to storage speed and the parity result is void. They gate on it at deployment time but don't pin or continuously monitor, so memory pressure in multi-tenant settings could silently nullify the benefit. It's disclosed, but it limits the deployment claim. Third, the GH200 sample is small and the Kimi K3 headline uses proprietary weights, so the most impressive number isn't independently reproducible. These are limitations, not errors.\n\nThis paper is for systems researchers who care about where bytes live in LLM inference on integrated or coherent-memory hardware. It deserves a serious referee. Recommendation: send to peer review, and make the artifact a condition of acceptance—the methodology is strong enough that the claims deserve independent reproduction.","headline":"Careful empirical systems work showing mapped file-backed weights can match resident bandwidth on unified/coherent memory at zero rate cost; the missing artifact is the main obstacle to unqualified acceptance.","tokens_in":28157,"tokens_out":4058,"would_cite":true,"duration_ms":33041,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Cached checkpoint pages can feed a GPU at resident speed if frameworks stop copying them first.","keywords":["file-backed weights","ingestion tax","memory mapping","page cache","GPU inference","unified memory","memory topology","tensor frameworks"],"falsifier":"On a unified-memory machine, run the paper's matched protocol with a fully cached checkpoint and a single kernel over the same bytes in two forms: imported mapped pages and one-time resident copies; the paper's central claim predicts the mapped arm stays within roughly one percent of the resident arm (the paper's interval is [-0.66%, +0.48%]), so an observed gap beyond that interval under the stated protocol would refute the no-rate-cost parity claim. A complementary control—forcing page-cache eviction mid-decode and watching the spine stage return to storage-bound time—would confirm the residency boundary rather than refute the mechanism.","tokens_in":27196,"feed_emoji":"⚡","tokens_out":10241,"duration_ms":90266,"temperature":0.7,"pith_summary":"Open-weight models often sit in a middle regime: the active weights fit in DRAM as cached file pages, but a second framework-owned copy does not fit or must be refilled every token. The paper isolates a cost it calls the ingestion tax—file-backed bytes the OS already holds in GPU-readable memory being copied into framework-owned allocations before each use—and argues that the tax is entirely avoidable on unified and coherent-memory systems. It presents file-backed weight adoption, a producer that maps each tensor with MAP_SHARED, wraps the pages as a no-copy GPU buffer, and exports an interchange capsule that tensor frameworks import as ordinary storage. Under a three-condition execution contract, the mapped path reaches 516 GB/s where default constructors reach 53–82 GB/s, and a matched 72B model decodes at essentially the same rate as a load-once resident control. The reason to care is that parity buys ownership: the weights stay one shared, evictable, reclaimable file-backed copy, which changes multi-process serving, startup latency, and the role of the page cache.","feed_headline":"Cached checkpoint pages can feed the GPU at resident speed","feed_subtitle":"Adopting file-backed weights as ordinary tensors removes the per-token copy tax on unified-memory machines.","key_machinery":"The mechanism is a framework-independent producer: it opens each page-aligned tensor file, maps it read-only with MAP_SHARED, wraps the range as a no-copy GPU buffer, and returns a versioned DLPack capsule—an open in-memory tensor interchange format—that a tensor framework imports as ordinary storage. The argument's second pillar is the three-condition execution contract: C1 adopt the mapped pages with no per-use copy, C2 keep activations in the accelerator-visible domain, and C3 establish dependencies on the GPU so the host never drains in-flight work. The paper prices each condition with a randomized same-harness factorial—removing C1, C2, or C3 costs 12.3, 5.6, or 3.4 ms per pass—which is what converts zero-copy import from a mechanism into a correct fast path; a deployment-time endpoint probe then chooses among map, resident-copy, and per-use stream-copy based on full-working-set bandwidth.","core_discovery":"The central claim is that on systems where file pages already occupy a GPU-readable memory domain, the only thing standing between a kernel and resident-class bandwidth is the framework's ownership model: constructor and transfer paths allocate a second representation and copy file-backed bytes into it, a pure software cost with no faster placement behind it. File-backed weight adoption removes that copy, and the paper shows the removal costs nothing in steady-state decode rate when three conditions hold: the kernel reads the mapped pages, activations stay accelerator-resident, and ordering happens on the GPU. The measured endpoint is 516 GB/s through the public import route against 53–82 GB/s for default constructors, with mapped-minus-resident bandwidth bounded by [-0.66%, +0.48%] and a matched 72B model at 7.14 versus 7.23 tokens/s relative to a load-once resident control. The same parity argument carries a deployment boundary: on a discrete PCIe GPU, mapped reads are interconnect-bound and adoption loses by 39x and should be refused, so memory topology, not the API, decides the byte path.","pith_inferences":["Beyond the paper: a natural extension is a residency-aware loading policy that combines adoption with cache-pinning or prefault hints, so the page-cache precondition is actively maintained instead of assumed.","Beyond the paper: the measured scaling with activation columns suggests the advantage shrinks as weight reuse grows, so fusing multiple tokens or heads into one kernel could preserve more of the benefit for batched or prefill regimes than the paper's low-batch setting demonstrates.","Beyond the paper: the read-only enforcement gap—a GPU store to the mapping is silently discarded on one substrate, faults on another, and rejected on a third—points toward a framework-level immutable external storage type; testing whether consumers reject in-place stores once immutability is propagated is a direct next experiment.","Beyond the paper: with the byte path decided by memory topology, the same endpoint probe could apply to emerging coherent-memory and CXL-style tiers, where mapped file pages might become the default accelerator placement the way they are on integrated parts."],"forward_implications":["On unified and coherent-memory systems, a checkpoint can be served as one clean, shared, evictable file-backed copy, so N concurrent decoders read the same pages instead of materialising N private copies.","Time-to-first-token for capacity-scale checkpoints improves sharply because module construction and shard parsing are replaced by mapping and first-touch wiring; the paper measures a 6.4x faster first token for a 65 GB checkpoint.","The same adoption mechanism improves an integrated-GPU inference runtime by 1.21x at about half the footprint, ties overlapped streaming on a coherent-link system whose model exceeds fast memory, and is rejected on a discrete PCIe GPU, so the deployment rule is topology- and working-set-aware.","Tensor frameworks need a first-class read-only mapped-storage constructor that handles alignment, lifetime, immutability, and placement; no new scheduler or intermediate representation is required.","The page cache becomes a reclaimable, shared accelerator-storage tier, so capacity checks should measure hot-set residency and representation count rather than model size against installed memory."],"supporting_citations":[{"why":"Establishes the standalone-runtime pattern of reading mapped weights without copying, which the paper generalizes to tensor-framework execution.","marker":"[15]"},{"why":"Supplies the documented zero-copy import surface that lets adopted capsules be consumed as ordinary storage.","marker":"[25]"},{"why":"Defines the interchange capsule format and its versioning and strides rules that the producer emits.","marker":"[10]"},{"why":"Reports the closest prior prototype that mapped model files at the allocator level and observed the page-cache collapse that marks the regime boundary.","marker":"[23]"},{"why":"Converges on the one-file-per-tensor layout and documents low-RAM streaming constraints that the adoption path inherits.","marker":"[24]"},{"why":"Supplies the coherent-link data-movement measurements that anchor the coherent-memory placement classes and the migration trajectory.","marker":"[12]"},{"why":"Shows that moving a memory-mapped CPU tensor to an accelerator is a separate placement step, the gap the producer closes.","marker":"[28]"}],"fun_headline_variants":["File-backed weights: zero-copy import hits 516 GB/s on unified memory","Skip the ingestion tax: file pages feed GPU at resident speed","No-copy weight adoption ties resident bandwidth on coherent memory","The ingestion tax is pure software; file-backed import removes it","Memory topology decides: zero-copy adoption on shared, refuse on PCIe"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The active mapped weight set must remain resident in the page cache for the whole decode; once the OS evicts those pages, storage traffic dominates and the in-memory parity result no longer applies.","fun_headline_variants_meta":{"raw":{"variants":["File-backed weights: zero-copy import hits 516 GB/s on unified memory","Skip the ingestion tax: file pages feed GPU at resident speed","No-copy weight adoption ties resident bandwidth on coherent memory","The ingestion tax is pure software; file-backed import removes it","Memory topology decides: zero-copy adoption on shared, refuse on PCIe"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000444,"raw_usage":{"total_tokens":2368,"prompt_tokens":1191,"completion_tokens":1177,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":807,"completion_tokens_details":{"reasoning_tokens":1085}},"tokens_in":807,"tokens_out":1177,"duration_ms":10865,"temperature":1.0,"reasoning_tokens":1085,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T00:15:29.694097+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a unified-memory machine, run the paper's matched protocol with a fully cached checkpoint and a single kernel over the same bytes in two forms: imported mapped pages and one-time resident copies; the paper's central claim predicts the mapped arm stays within roughly one percent of the resident arm (the paper's interval is [-0.66%, +0.48%]), so an observed gap beyond that interval under the stated protocol would refute the no-rate-cost parity claim. A complementary control—forcing page-cache eviction mid-decode and watching the spine stage return to storage-bound time—would confirm the residency boundary rather than refute the mechanism.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes the standalone-runtime pattern of reading mapped weights without copying, which the paper generalizes to tensor-framework execution."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the documented zero-copy import surface that lets adopted capsules be consumed as ordinary storage."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the interchange capsule format and its versioning and strides rules that the producer emits."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Reports the closest prior prototype that mapped model files at the allocator level and observed the page-cache collapse that marks the regime boundary."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Converges on the one-file-per-tensor layout and documents low-RAM streaming constraints that the adoption path inherits."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Shows that moving a memory-mapped CPU tensor to an accelerator is a separate placement step, the gap the producer closes."}],"review_version":1}