{"id":"8ffdbebe-377b-45ab-93a7-a483e789376c","arxiv_id":"2608.06989","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A dual-view memory system with offline joint mapping optimization and runtime accessor-aware scheduling achieves up to 2.32x higher LLM inference throughput than prior unified NPU-PIM memory designs in simulation.","lead":"This paper proposes PFM, a memory controller design that lets NPU and processing-in-memory (PIM) units share the same physical DRAM while each sees a tailored data layout. If the simulation results hold on real hardware, it could remove a major bandwidth bottleneck in NPU-PIM systems running large language models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Equation (7) divides by N_row (rows per bank) instead of N_col (columns per row) when estimating PIM row-activation time, so the mapping search that selects PFM's joint layouts is solving a miscalibrated objective; the claimed near-peak PIM utilization and 2.32x speedup are therefore not…","rationale":"The reader's weakest assumption was that the analytical bandwidth model (Eqs. 2-9) accurately predicts real HBM-PIM bandwidth for mixed NPU/PIM access; my review identifies a concrete internal error inside that model. Eq. (7) misuses N_row as the divisor for converting data volume into row activations, when the correct quantity is the number of columns per row, N_col. This error propagates into Eq. (5) and into the mapping optimization that is the core novelty of PFM. The final Ramulator-based cycle-level evaluation is a real strength and partially mitigates the concern: it would expose timing violations for a given mapping. However, it cannot correct the search: the optimizer is not rerun under the corrected objective, so the paper's assertion that PFM 'solves a multi-objective optimization problem' to derive the optimal layout is not supported by the equations as written. I also note the secondary inconsistency between the abstract's 2.32x and the conclusion's 2.26x speedup, but that is not the load-bearing issue. Because the reader already conditioned acceptance on validating the bandwidth model, my finding reinforces that condition rather than changing the verdict; hence UNCHANGED. No ad hominem is intended; the issue is purely in the technical derivation.","tokens_in":23727,"tokens_out":7135,"duration_ms":80554,"concrete_test":"Independently re-derive Eq. (7) from DRAM geometry: row activations per bank = (2^{p_n+p_k}/#active_banks) / N_col, and re-run the COIN-OR/PuLP mapping search for the Mixtral 4096x14336 expert-weight tensor and for GPT-OSS KV-cache blocks with the corrected T_ref. Then re-evaluate the chosen mappings in the Ramulator 2.0 HBM-PIM model and compare PIM bandwidth utilization (Table III) and end-to-end throughput (Figure 13) against the paper's numbers. If the corrected search changes the selected MapIDs or lowers reported PIM utilization by more than about 5 percentage points, the claim that PFM's layout is jointly optimized is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim depends on the offline dual-view mapping optimizer (Section IV-B) choosing a layout that is simultaneously good for NPU and PIM. The PIM objective PIM(f) in Eq. (5) uses T_ref from Eq. (7), defined as T_ref = (2^{p_n+p_k}/(#active_banks(f) * N_row)) * t_RCD + N_col * t_CCD, where N_row is the number of rows per bank and N_col is the number of columns per row. To estimate row activations per bank, the bank's data volume (2^{p_n+p_k}/#active_banks) must be divided by the number of elements per row, which is N_col, not by N_row. As written, for a 2 MB page on a 1024-bank HBM stack with N_row ~16K and N_col ~32, the first term is undercounted by roughly 512x. Since T_ref is in the denominator of Eq. (5), the optimizer is solving a different problem than the one claimed. The later Ramulator evaluation (Section VII-A) can validate the timing of a chosen mapping, but it does not repair the mapping selection itself: if the search was biased by the faulty T_ref, the solver may return layouts that are not jointly optimal. The reported 94.85% PIM bandwidth utilization (Table III) and the up-to-2.32x end-to-end speedup (Section VII-B) therefore rest on an internally inconsistent analytical model. This is a concrete flaw in the paper's own equations, independent of whether the simulator is accurate.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes PFM (PIM-as-Flexible-Memory), a dual-view memory system for NPU-PIM architectures that decouples physical data layout from accessor-visible logical views. PFM stores tensors in a single jointly optimized physical layout and exposes different address interpretations to the NPU and PIM, supported by an Address Remapping Unit (ARU) and a Flexible Access Scheduler (FAS). An offline mapping optimizer (Eq. 10) searches over address bit assignments, and a runtime roofline model (Eqs. 11-12) assigns each operator to the NPU or PIM. The paper evaluates PFM with a GPGPU-Sim/Ramulator simulator across LLaMA3, DeepSeekMoE, Mixtral, and GPT-OSS, reporting up to 2.32x throughput improvement over a PIM-friendly unified-memory baseline (PUM) and near-peak bandwidth utilization for both devices (92.1% NPU, 94.85% PIM).","tokens_in":24147,"tokens_out":11159,"duration_ms":110597,"significance":"The problem is timely and real: existing unified-memory designs assume static device-biased tensors, which mismatches the dynamic device preferences of MoE routing and prefill/decode phase changes in LLM inference. The dual-view concept is a plausible and potentially valuable alternative to data migration or duplication. The paper includes a detailed memory-controller design, an analytical model, a case study, and an area-overhead estimate, and it ships real MoE activation traces. However, the central quantitative claims rest on an analytical model that appears to contain a dimensional error, and on several offline-fitted parameters without sensitivity analysis. If these issues are corrected and the results remain robust, the contribution could be significant.","major_comments":[{"comment":"Eq. (7) defines T_ref with the first term divided by N_row (rows per bank), but the number of row activations per bank for a data volume of 2^{p_n+p_k}/#active_banks(f) elements should be that volume divided by N_col (elements per row). For the evaluated HBM configuration (2 MB page, 1024 banks, N_row ~16K, N_col ~32), the first term is undercounted by roughly N_row/N_col, i.e., about 512x. Since T_ref appears in the numerator of PIM(f) in Eq. (5), the PIM objective loses its intended dependence on per-bank data volume and becomes numerically negligible relative to NPU(f) in Eq. (10). The offline mapping search therefore is not solving the claimed joint dual-view optimization, and the selected layouts are not demonstrated to be jointly optimal. The Ramulator evaluation in Section VII-A cannot repair this because it only validates the timing of a given mapping. Please correct Eq. (7) (replace N_row with N_col) and re-run the mapping search; if the selected mappings and reported speedups are unchanged, state that explicitly with evidence.","section":"IV-B, Eq. (7)"},{"comment":"The runtime device-selection model uses gamma and T_setup that are determined offline (Section V-B), and the mapping optimizer uses alpha chosen through profiling (Section IV-B4). The paper does not report the numerical values of alpha, gamma, and T_setup, nor any sensitivity analysis. Because the end-to-end speedups depend on the scheduler decisions made by this model, and because the simulator and the scheduling model share the same analytical model family, the reported gains could be partly an artifact of parameter fitting. Please report the fitted values and perform a sensitivity analysis (e.g., vary each parameter over a plausible range and report the resulting throughput). In addition, validate the predicted PIM execution times for the selected mappings directly against the detailed Ramulator timing model.","section":"V-B (Eqs. 11-12), IV-B4"},{"comment":"The abstract claims 'up to 2.32x end-to-end throughput over prior unified memory designs,' but Section VII-A states that in large-batch settings, the most flexible prior unified design (FACIL) is equivalent to PSM, and the speedup over PSM is up to 2.29x (Figure 13). The 2.32x figure is over PUM, an all-PIM-friendly layout that is not representative of the strongest prior unified design in large-batch settings. Please qualify the headline claim or present the comparison against the strongest baseline throughout the paper.","section":"Abstract, VII-B"},{"comment":"The simulator is validated against real A100 measurements for the NPU side, but the HBM-PIM timing model is not validated against any real PIM hardware. Since the claimed 94.85% PIM bandwidth utilization (Table III) is a central result, please provide additional evidence that the modeled PIM timing constraints (bank-locking, SIMD width, command scheduling) match a published HBM-PIM implementation, or at least discuss how deviations from real hardware would affect the conclusions.","section":"VII-A"}],"minor_comments":[{"comment":"The notation Ppn−1j=0 X_row,j would benefit from an explicit definition of X_row,j as the entry of the mapping matrix X corresponding to the row level and address bit j; currently the index convention is unclear.","section":"IV-B, Eq. (4)"},{"comment":"The bit ranges [11:7], [20:19], and similar in the text are not labeled directly on the mapping diagram; please add the bit ranges to the figure or caption for readability.","section":"VI-A, Figure 12"},{"comment":"The description of the PSM baseline as 'NeuPIMs-like' is vague; please state whether migration cost between the separated NPU and PIM regions is modeled, as this affects the baseline's absolute performance.","section":"VII-A"},{"comment":"The 'Oracle' strategy is described only qualitatively; please specify whether it assumes zero address-translation and scheduling overhead and whether it can re-optimize the layout per operator without cost.","section":"III-A"},{"comment":"The abstract reports 'up to 2.32x' improvement while the conclusion reports 'up to 2.26x'; please reconcile these numbers.","section":"Abstract vs. IX"},{"comment":"The Limitations paragraph lists re-profiling and MDT size, but it does not mention the sensitivity of the results to the alpha/gamma/T_setup parameters; we recommend adding a sentence acknowledging this dependency.","section":"VIII"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely and important problem, and the dual-view concept is a genuine step beyond static device-biased unified memory. However, the apparent dimensional error in Eq. (7) affects the claimed joint optimization, and the lack of sensitivity analysis for alpha, gamma, and T_setup makes it hard to separate the contribution of the design from the fitted parameters. I recommend major revision with a request to correct Eq. (7), re-run the mapping search, and add sensitivity/validation analyses. If the results remain robust, the paper could become a strong contribution; the headline speedup should also be stated with respect to the strongest baseline."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know up front. First, this paper has a real and useful idea: instead of binding tensors to a device-biased layout, give each accessor a different logical view of one jointly optimized physical layout, and let the memory controller schedule both streams. That is a clean answer to the dynamic-device problem in MoE-style LLM inference, and it goes beyond IANUS and FACIL. Second, the paper's central results—92-95% bandwidth utilization and up to 2.32x speedup—rest on an analytical model that contains a concrete error. Equation (7) divides by N_row (rows per bank) where it should divide by N_col (columns per row) to get row activations per bank. That makes T_ref far too large, so PIM(f) is inflated and the mapping solver is optimizing a miscalibrated objective. The Ramulator experiments can validate the timing of a chosen mapping, but they cannot repair the search that picked that mapping. The numbers may survive the fix, but the paper as written does not establish that they do.\n\nWhat is legitimately good: the problem statement is well documented (real MoE expert traces, quantitative comparison to oracle), the dual-view address translation and scheduling are concrete, and the evaluation is unusually complete for this subfield—four LLMs, multiple baselines, A100 timing validation, and RTL synthesis for the added controller logic. They also acknowledge the profiling/MDT limitation in the discussion.\n\nSoft spots, in proportion: the Eq. (7) error is the load-bearing one. The 2.32x in the abstract vs 2.26x in the conclusion is a trivial inconsistency. alpha, gamma, and T_setup are fit offline with no sensitivity analysis; the simulator shares the same model as the optimizer, so some circularity is unavoidable. None of these are fatal to the concept, but they matter for the claimed speedups.\n\nThis paper deserves a serious referee. The idea is novel and likely worth building on. But I would not cite its numbers until Eq. (7) is corrected and the mapping search re-run. If the corrected results hold, this is a strong systems contribution.","headline":"A genuinely new mechanism for NPU-PIM memory, with a thorough evaluation, but Eq. (7) has a units error that biases the mapping search; numbers should not be trusted until that is corrected.","tokens_in":24585,"tokens_out":5388,"would_cite":true,"duration_ms":54407,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PFM: a dual-view memory design that lets NPU and PIM share one physical layout with device-specific logical views, enabling dynamic LLM inference without duplication or relayout.","keywords":["processing-in-memory","NPU-PIM systems","unified memory","LLM inference","dual-view memory","address remapping","memory scheduling","Mixture-of-Experts"],"falsifier":"Measure, on real HBM-PIM hardware or a validated cycle-accurate simulator, the achieved DRAM bandwidth for the specific superpage mappings PFM's solver selects under NPU-only, PIM-only, and mixed NPU/PIM request streams, and compare those measurements with the model's predicted $B_{\\rm NPU}$ and $B_{\\rm PIM}$ values. If the balanced layouts systematically fall well short of the predicted 92.1% NPU and 94.85% PIM utilization, the mapping optimizer and the 2.32x end-to-end claim would not carry over to hardware.","tokens_in":23558,"feed_emoji":"🔀","tokens_out":7667,"duration_ms":69816,"temperature":0.7,"pith_summary":"The paper argues that existing unified memory designs for NPU-PIM systems fail on modern LLM inference because they bind each tensor to one device at allocation time, while LLM workloads like MoE routing and prefill/decode shifts make the best device change at runtime. It proposes PFM, which stores each tensor once in a jointly optimized physical layout and gives NPU and PIM different logical views of the same data, so both can access it at high bandwidth without duplication or relayout. A memory-controller address remapping unit and flexible scheduler implement the dual views, and offline multi-objective optimization over 2 MB superpages chooses the physical mapping. The paper reports up to 2.32x end-to-end throughput over the best prior unified memory baseline and near-peak bandwidth utilization on both devices. If right, PFM removes the static-allocation assumption that limits NPU-PIM memory systems for dynamic LLM serving.","feed_headline":"Dual-view memory speeds NPU-PIM LLM inference up to 2.32x","feed_subtitle":"PFM stores each tensor once and lets NPU and PIM read it in their preferred layout.","key_machinery":"The load-bearing object is the per-superpage address-bit assignment matrix $X \\in \\{0,1\\}^{m\\times n}$, which says which physical-address bits go to which DRAM level (channel, bank group, bank, row, column). PFM solves $\\max_f \\text{NPU}(f)+\\alpha\\,\\text{PIM}(f)$ for each tensor's 2 MB superpage, where $\\text{NPU}(f)$ estimates achievable bandwidth from row-activation and column counts and $\\text{PIM}(f)$ estimates bank-level parallelism gain from the number of active banks and intra-bank access time. The resulting mapping is stored as a MapID descriptor, and at runtime the Address Remapping Unit applies it to translate physical addresses into hardware addresses in an accessor-specific way. The paper's case study shows the optimizer placing channel bits low and row bits high so a single Mixtral expert weight page is channel-parallel for the NPU and bank-local for PIM.","core_discovery":"The central claim is that a fixed physical layout can serve two devices with conflicting address-mapping preferences if each device reads it through its own logical view. PFM stores LLM tensors in a physical layout chosen offline by solving a per-superpage multi-objective problem that balances the NPU's need for channel interleaving against the PIM's need for bank-local continuity and parallelism. A memory-controller Address Remapping Unit then translates the same physical address differently for NPU and PIM requests, and a Flexible Access Scheduler reorders NPU requests for channel concurrency and row hits while batching PIM requests for bank-parallel broadcasts. The paper reports that on Mixtral this yields 92.1% NPU and 94.85% PIM memory bandwidth utilization, and that across LLaMA3-8B, DeepSeekMoE-16B, Mixtral-8x7B, and GPT-OSS-120B it improves end-to-end throughput by up to 2.32x over the best prior unified-memory design (PUM), reaching 96.8% of an oracle's decode efficiency.","pith_inferences":["Beyond the paper, the dual-view abstraction should transfer to any heterogeneous system where the same data is consumed by accessors with conflicting layout preferences, such as GPU-CPU or GPU-DPU pairs, not only NPU-PIM.","A testable extension is to close the loop online: instead of requiring offline re-profiling when expert routing shifts, the MapID table could be repopulated from measured access streams, turning the mapping search into a continuous optimizer.","The paper's offline solver takes up to 30 minutes per tensor, so serving many concurrently changing model variants would need cheaper heuristics or reuse of MapIDs across shape classes; the paper assumes limited map templates suffice for common deployments.","Because the runtime scheduler prioritizes NPU bursts and drains PIM during NPU compute windows, the design could also produce predictable QoS for latency-sensitive LLM serving, though the paper does not report tail-latency or multi-tenant contention results."],"forward_implications":["If PFM is correct, LLM operators can be re-dispatched between NPU and PIM at decoding-step granularity without paying for data migration or layout conversion.","MoE expert weights and KV caches can occupy one shared physical address space and still deliver near-peak bandwidth to both NPU and PIM, which is what drives the 2.32x end-to-end throughput improvement.","The static device-biased mappings of prior unified memory designs leave most of the oracle speedup unrealized; FACIL captures only 29.5% of the oracle gain, while PFM's dual-view design reaches 96.8% of oracle efficiency in decode.","Because only the memory controller is modified, the same dual-view approach can be applied to HBM, GDDR, and LPDDR systems and is independent of the NPU architecture."],"supporting_citations":[{"why":"FACIL, the flexible DRAM address mapping design for SoC-PIM; it supplies the device-biased unified-memory baseline and the address-mapping approach PFM extends.","marker":"[49]"},{"why":"IANUS, the NPU-PIM unified memory system with a globally PIM-friendly layout; it represents the static unified-memory alternative PFM contrasts with.","marker":"[48]"},{"why":"AttAcc, the batched transformer PIM inference design; PFM uses its near-bank PIM organization, trace-generation method, and PIM execution constraints.","marker":"[41]"},{"why":"NeuPIMs, the separated-memory NPU-PIM system; it is the PSM baseline with static NPU/PIM region allocation.","marker":"[17]"},{"why":"vLLM and PagedAttention; they define the serving workload, KV-cache block management, and the 2 MB superpage compatibility target.","marker":"[25]"},{"why":"Mixtral 8x7B; it is the MoE model used for the detailed expert-weight case study and part of the evaluation set.","marker":"[20]"},{"why":"GPT-OSS-120B; with LMSYS-Chat-1M traces it supplies the real-world expert activation patterns used in the motivation and throughput experiments.","marker":"[39]"},{"why":"The AiM GDDR6 accelerator-in-memory design; it provides the PIMMAC instruction encoding and PIM device parameters PFM assumes.","marker":"[27]"}],"fun_headline_variants":["Dual-view memory speeds NPU-PIM LLM inference up to 2.32x","One layout, two views: 2.32x faster LLM inference on NPU-PIM","PIM-as-Flexible-Memory: dual-view design eliminates duplication, 2.32x speedup","Dynamic LLM execution gets 2.32x boost with dual-view memory for NPU-PIM"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The result depends on the assumption that the paper's equations for estimating memory bandwidth closely match what real HBM-PIM hardware delivers for the mixed NPU and PIM access patterns it chooses.","fun_headline_variants_meta":{"raw":{"variants":["Dual-view memory speeds NPU-PIM LLM inference up to 2.32x","One layout, two views: 2.32x faster LLM inference on NPU-PIM","PIM-as-Flexible-Memory: dual-view design eliminates duplication, 2.32x speedup","Dynamic LLM execution gets 2.32x boost with dual-view memory for NPU-PIM"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001102,"raw_usage":{"total_tokens":4653,"prompt_tokens":1060,"completion_tokens":3593,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":676,"completion_tokens_details":{"reasoning_tokens":3489}},"tokens_in":676,"tokens_out":3593,"duration_ms":24620,"temperature":1.0,"reasoning_tokens":3489,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T17:00:32.907131+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure, on real HBM-PIM hardware or a validated cycle-accurate simulator, the achieved DRAM bandwidth for the specific superpage mappings PFM's solver selects under NPU-only, PIM-only, and mixed NPU/PIM request streams, and compare those measurements with the model's predicted $B_{\\rm NPU}$ and $B_{\\rm PIM}$ values. If the balanced layouts systematically fall well short of the predicted 92.1% NPU and 94.85% PIM utilization, the mapping optimizer and the 2.32x end-to-end claim would not carry over to hardware.","supporting_citations":[{"cited_title":"Ianus: Integrated accelerator based on npu-pim unified memory system,","cited_arxiv_id":null,"evidence_quote":"IANUS, the NPU-PIM unified memory system with a globally PIM-friendly layout; it represents the static unified-memory alternative PFM contrasts with."},{"cited_title":"Neupims: Npu-pim heterogeneous acceleration for batched llm inferencing,","cited_arxiv_id":null,"evidence_quote":"NeuPIMs, the separated-memory NPU-PIM system; it is the PSM baseline with static NPU/PIM region allocation."},{"cited_title":"Gpt-oss-120b,","cited_arxiv_id":null,"evidence_quote":"GPT-OSS-120B; with LMSYS-Chat-1M traces it supplies the real-world expert activation patterns used in the motivation and throughput experiments."}],"review_version":1}