{"id":"5df286ab-3b84-4af4-96c4-d3432e69091f","arxiv_id":"2505.00901","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"MemScope is a kernel-level toolkit that benchmarks heterogeneous memories in multicore embedded systems under configurable contention, and it reveals new cross-memory interference effects.","lead":"This paper introduces MemScope, an open-source Linux kernel module for benchmarking heterogeneous memories under controlled contention. It gives embedded systems developers kernel-level control to measure bandwidth and latency of DRAM, FPGA DRAM, BRAM, and on-chip memory in multicore systems.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Mempool apertures are not shown to be isolated from the Linux memory map; if the DTB memory nodes in Fig. 15 are also registered as system RAM, MemScope benchmarks kernel-owned memory rather than the intended heterogeneous pool.","rationale":"The reader's weakest assumption identifies the same load-bearing point: the benchmark is only meaningful if the mempool apertures are genuine, private, correctly-accessed physical memory. I considered two other candidates. The MLP derivation in Section IV-B(3) pairs latency and bandwidth measurements from different access streams and could bias the MLP estimates, but it is downstream of pool correctness. The validation in Section IV-D compares only LLC-hit bandwidth against IsolBench, so the DRAM/PL-DRAM/BRAM/OCM curves are not independently calibrated; again, this matters most if the wrong aperture is being measured. The DTS in Figure 15 is explicit in-text evidence for the overlap risk, because device_type='memory' is the standard ARM64 registration for RAM, and the paper does not document any reservation mechanism. The memremap attribute issue compounds this: without stating whether the mapping is WB, WT, or device-nGnRnE, the effect of cache maintenance instructions in the non-cacheable workloads is unquantified. Both parts of the concern are resolvable from the released code and boot configuration, so the appropriate disposition remains conditional rather than accept or reject.","tokens_in":23405,"tokens_out":12422,"duration_ms":136479,"concrete_test":"Release the repository with the exact DTS and kernel command line, boot the ZCU102, load MEMSCOPE, and compare the physical ranges reported by the pools debugfs entry with /proc/iomem and dmesg reservations. If any DRAM/PL-DRAM pool range appears as System RAM without a matching reserved/no-map entry, the pool is not isolated and the reported bandwidth/latency curves are invalidated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III-B and Appendix A-A create each pool from any DTB node with compatible='mempool' by calling memremap and adding the resulting KVA to a gen_pool. The paper never describes reserving these apertures from the kernel's normal memory allocator (no reserved-memory/no-map, no mem= carve-out). Figure 15 is the problem: it shows dram@10000000 with device_type='memory' and a 256 MB range, which is exactly the node type ARM64 Linux uses to register system RAM. Unless that range is excluded elsewhere, gen_pool_alloc can hand out pages that the kernel or other processes are using, so the 'target memory' is not an isolated pool and the benchmark may read/write live kernel data. The memremap flags are also unspecified; if the mapping is WB, normal r/w workloads on a 32 KB OCM/BRAM buffer hit the cache, and the non-cacheable variants in Appendix A-B are cacheable loads/stores followed by dc civac, not an actually uncached mapping. Every bandwidth and latency result depends on the pool being the intended, private aperture, so this is the load-bearing uncertainty.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents MemScope, a Linux kernel module for benchmarking heterogeneous memories (PS-DRAM, PL-DRAM, OCM, BRAM) on an ARM64 embedded platform. MemScope discovers memory regions from device tree nodes with a custom compatible string, creates gen_pool-based allocators per region, and provides a workload library for bandwidth and latency measurements with configurable multi-core contention. The evaluation on a Xilinx ZCU102 characterizes DRAM variants and scratchpad memories, derives memory-level parallelism via Little's Law, reproduces a known cache-bank contention effect, validates bandwidth against IsolBench, and demonstrates application-level allocation insights using RT-Bench. The central claim is that MemScope is the first kernel-level toolkit for heterogeneous memory characterization in multicore systems, enabling precise measurements with kernel-level control over allocation, cache state, interrupts, and CPU activity.","tokens_in":23640,"tokens_out":7065,"duration_ms":74605,"significance":"If the results are valid, MemScope would be a valuable open-source contribution to the real-time embedded systems community, addressing genuine limitations of user-space benchmarking (C1–C5). The validation against IsolBench (Figure 10) and the reproduction of cache-bank contention on a Cortex-A53 platform (Section IV-D) are credible and demonstrate the framework's capability to capture known microarchitectural effects. The kernel-level design with explicit multi-core synchronization and performance-counter sampling is a useful engineering effort. However, the validity of all quantitative results depends critically on the correctness of the memory-pool isolation mechanism, which the manuscript does not adequately describe.","major_comments":[{"comment":"The design does not describe how the memory apertures are excluded from the kernel's normal memory management. The DTS snippet in Figure 15 declares dram@10000000 with device_type=\"memory\" and a 256 MB range; on ARM64 Linux this is exactly the node type used to register system RAM. If this range is also part of the kernel's memory map, then pages allocated from the gen_pool created by memremap can overlap with pages owned by the buddy allocator, and the benchmark may read/write live kernel data. The paper only states that these are \"the slices we carve out\" (Section IV-A) but provides no reserved-memory node, no \"no-map\" property, no mem= kernel parameter, or any other mechanism that would remove these ranges from the kernel's memory map. Establishing this reservation is mandatory before any of the reported bandwidth or latency numbers for DRAM and PL-DRAM can be accepted. Please describe the exact boot configuration (DTB, kernel command line, or U-Boot script) that ensures these apertures are private to MemScope.","section":"Section III-B, Appendix A-A, Figure 15"},{"comment":"The memremap flags are unspecified, which leaves the interpretation of the \"non-cacheable\" workloads ambiguous. If the apertures are mapped with MEMREMAP_WB (or remain in the direct map), then the supposedly non-cacheable operations (s, x, m) are actually cacheable accesses followed by dc civac, not truly uncached accesses. The first access in each loop iteration can hit in cache, and the effective mapping determines whether the measurement reflects the target memory or the cache hierarchy. Section III-E states that only normal cacheable mappings are considered, which is at odds with the non-cacheable claims in Section IV-C. Please specify the exact memremap flags used for each memory pool, and explain how the measured scratchpad results for OCM and BRAM (Figures 8 and 9) correspond to the memory module itself rather than to cached or device-mapped behavior.","section":"Appendix A-A, Appendix A-B"},{"comment":"The MLP calculation via Little's Law uses a latency from experiments with access pattern (l,r) or (l,w) and a bandwidth from experiments with access pattern (r,r) or (r,w). These are not the same workload on the core under observation: the latency experiment uses dependent pointer-chasing loads, while the bandwidth experiment uses sequential reads. Little's Law requires steady-state latency and bandwidth for the same traffic mix. Without a consistent pairing, the derived MLP values are not a well-defined property of the memory subsystem. Since the subsequent conclusion about a shared-bus MLP bottleneck (Section IV-B(4)) rests on the validity of these MLP numbers, the pairing must be justified or the experiments redesigned so that latency and bandwidth are measured under the same access pattern.","section":"Section IV-B(3), Tables II-III"}],"minor_comments":[{"comment":"The paper reports 500 iterations per scenario but no error bars, confidence intervals, or standard deviations. For a benchmarking toolkit that claims precision, reporting at least the median and the interquartile range for the latency and bandwidth results would strengthen statistical credibility.","section":"Section IV-A, Figures 4-13"},{"comment":"In the comparison table, the entries for LENS and Heimdall appear to indicate that these tools run in kernel mode with a check mark, which contradicts the statement in Section V that \"the majority are implemented in user space.\" Please verify the column semantics and the correctness of these entries.","section":"Table V"},{"comment":"The latency workload is described as \"impossible to prefetch,\" but the paper does not state whether hardware prefetchers are disabled or how the randomized permutation prevents prefetching. A brief explanation or a pointer to the code would be helpful.","section":"Section III-C"},{"comment":"The non-cacheable write strategies x and y are defined differently (store followed by dc civac, and dc zva write streaming, respectively). It would be clearer to explain in the experiments where each strategy is used, especially since the y strategy is used with w* notation in Figures 13.","section":"Appendix A-B"},{"comment":"The name of the toolkit is written both as \"MEMSCOPE\" and \"MemScope\" in the text and figures; please standardize the spelling.","section":"Throughout"},{"comment":"The claim that the toolkit \"drives memory allocation in user-space applications\" is supported only by the RT-Bench experiments in Figure 14 and Appendix B. The paper does not discuss the policies or heuristics that would actually make these allocation decisions, so the claim is stronger than the evidence presented.","section":"Section IV-E"}],"recommendation":"major_revision","confidential_remarks":"The most serious issue is the missing description of physical memory reservation. If the authors can demonstrate that the tested apertures are truly carved out from the kernel's memory map, the paper could be acceptable after revision; otherwise, the reported DRAM and PL-DRAM results are potentially sampling arbitrary kernel pages. I recommend that the editor ask specifically for the boot configuration (DTB, kernel command line, bootloader scripts) and for the memremap flags used. The MLP pairing issue is also worth a careful revision. The paper otherwise has solid engineering and a useful validation against IsolBench."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, this is a genuinely useful kernel-level toolkit for benchmarking heterogeneous memories on ARM MPSoCs — the first for multicore, as far as the comparison table suggests, and it ships as a loadable module with no kernel source changes. Second, the main experimental vulnerability is real: nowhere in the paper are the mempool apertures shown to be isolated from the Linux memory map, so the benchmarks may be writing to live kernel memory rather than the intended hardware pool.\n\nWhat is good. The design is sound: device-tree based pool discovery, gen_pool allocation, assembly microbenchmarks, and careful multi-core synchronization via spinlocks. Validation against IsolBench is honest and the numbers line up well. Reproducing the cache bank contention under hits on an in-order Cortex-A53, where it was previously shown on out-of-order cores, is a solid new data point. The cross-memory interference result — DRAM latency degrading when stressors target PL-DRAM, attributed to shared bus queues — is non-obvious and well argued. The citation pattern is fine and the related work table is accurate.\n\nWhere the soft spots are. The pool reservation issue is the big one. Figure 15 shows dram@10000000 with device_type=\"memory\" — that is exactly what ARM64 Linux registers as system RAM. The paper says these are \"slices we carve out\" but never describes the carve-out: no reserved-memory, no no-map, no mem= boot argument. If the ranges are still system RAM, gen_pool_alloc can hand out pages the kernel or other processes are using, and every bandwidth and latency number in the paper is measuring the wrong thing. The memremap flags are also unspecified; if the mapping is write-back, the \"non-cacheable\" workloads in the appendix are just cacheable loads/stores followed by dc civac, which still approximates a miss stream but includes invalidation overhead in the reported numbers.\n\nMinor issues that should be fixed but are not fatal: no error bars or confidence intervals despite 500 iterations; the MLP derivation pairs latency experiments with bandwidth experiments from different runs (reasonable but approximate); and the open-source repository is not linked, despite the open-source claim. None of these compare to the reservation gap.\n\nWho this is for and what to do. Real-time and embedded systems researchers studying memory interference will get value from the toolkit and the empirical observations. The paper deserves a serious referee — this is a legitimate contribution that should be revised, not desk-rejected. But the revision must add an explicit, testable description of how the apertures are excluded from the kernel allocator (a snippet of the real DTS with reserved-memory/no-map, or the boot command line), provide the code repository with a commit hash, and add variance information. Until then, the central numbers are conditional.","headline":"Useful kernel-level heterogeneous memory benchmarking toolkit with a real validation story, but the paper never shows the benchmark pools are carved out of system RAM — a load-bearing gap that needs fixing before the numbers can be trusted.","tokens_in":24133,"tokens_out":3493,"would_cite":false,"duration_ms":37497,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"MEMSCOPE moves memory benchmarking into the Linux kernel, where a designated core can measure a memory module's bandwidth and latency while other cores inject controlled contention, yielding heterogeneous-memory characterizations that…","keywords":["heterogeneous memory","kernel-level benchmarking","memory contention","bandwidth and latency measurement","memory-level parallelism","FPGA-side DRAM","BRAM","real-time embedded systems"],"falsifier":"If a pool is declared with a device-tree node that points at an ordinary cached region rather than a dedicated uncached aperture, and MEMSCOPE still reports clean, low-variance results whose latency matches cache-hit latency instead of the memory module's true latency, then the memremap-based targeting is not actually measuring the intended memory.","tokens_in":23235,"feed_emoji":"📊","tokens_out":7470,"duration_ms":73133,"temperature":0.7,"pith_summary":"MEMSCOPE is a Linux kernel module that benchmarks heterogeneous memory subsystems in embedded multicore systems by moving measurement out of user space. The paper claims that only kernel-level control over physical memory allocation, cache maintenance, CPU state, and interrupts can produce contention-controlled, repeatable bandwidth and latency numbers across DRAM, FPGA-side DRAM, on-chip memory, and block RAM. On a Zynq UltraScale+ platform, the toolkit produces measurements that match an established user-space benchmark while also exposing effects user-space tools miss, such as FPGA-side DRAM traffic degrading CPU-side DRAM throughput through shared bus queue occupancy. The larger claim is that these precise measurements can directly guide memory allocation choices for real-time applications.","feed_headline":"In-kernel benchmark reveals how FPGA-side DRAM throttles CPU DRAM","feed_subtitle":"A Linux kernel module measures bandwidth and latency of DRAM, BRAM, OCM, and PL-DRAM under controllable multicore contention.","key_machinery":"The load-bearing object is the pairing of a device-tree-driven memory pool manager with a spinlock-synchronized core coordinator. Each hardware memory module is exposed in the device tree as a mempool node; MEMSCOPE maps that aperture with memremap and registers it as a genalloc pool, so a pool ID names one physical memory. The core coordinator sandwiches the observed core's measurement between two spin phases, guaranteeing that all stressor and idle cores have started before timing begins and have stopped before the next scenario. The workload library supplies assembly-coded sequential accesses for bandwidth and a randomized data-dependent pointer chain for latency, plus non-cacheable variants that flush or bypass caches, so the measured quantity is the target memory rather than the cache.","core_discovery":"The central claim is that MEMSCOPE is the first kernel-level toolkit for characterizing heterogeneous memory in multicore systems, and that this level of privilege changes what can be measured. The paper argues that user-space benchmarking cannot control which physical memory a buffer lands in, cannot keep daemons and interrupts off the measured core, and cannot flush caches reliably, so noisy results obscure the true temporal behavior of each memory module. MEMSCOPE answers by turning device-tree memory nodes into allocatable kernel memory pools, running assembly microbenchmarks on a designated observed core, and synchronizing stressor cores with spinlocks so the measurement window excludes partial overlap. Its evaluations characterize DRAM, PL-DRAM, OCM, and BRAM under escalating contention; derive memory-level parallelism via Little's Law; reproduce known cache bank contention effects; and show that a streaming-write stressor can degrade a protected core's bandwidth by roughly 40 times.","pith_inferences":["The spinlock-sandwich synchronization pattern could be factored into a general kernel service for profiling memory quality-of-service, letting the operating system build per-application memory-sensitivity maps at runtime without recompiling benchmarks.","A direct testable extension of the bus-queue-occupancy story is that an interconnect-level bandwidth regulator should protect CPU-side DRAM from FPGA-side DRAM traffic more effectively than per-bank DRAM throttling; the paper does not implement such a regulator.","The randomized pointer-chain latency workload, being designed to defeat prefetching, could double as a calibration harness to isolate prefetch-related effects on other platforms.","The device-tree pool mechanism suggests a spatial-isolation use beyond benchmarking: kernel-enforced memory pools could confine safety-critical tasks to dedicated memory, though the paper does not evaluate integrity or security properties."],"forward_implications":["A user-space application can map its heap to a specific memory pool through the exported device files, so the characterization data can be turned directly into an allocation decision.","The measured memory-level parallelism values, around 4 to 5 for both DRAM types, imply that the shared bus infrastructure is the bottleneck; inter-memory interference is therefore a first-order concern for memory allocation.","Cache partitioning reserves cache space but does not remove hit-path bank contention, meaning isolation mechanisms must address the bank level to be effective.","The escalating-stress experiment design supplies a worst-case performance envelope that is directly useful for schedulability analysis in real-time systems.","Because pool detection is device-tree driven, adding a new memory technology such as non-volatile or remote memory requires only a new device-tree node rather than new benchmarking code."],"supporting_citations":[{"why":"Establishes the precedent of kernel-level microbenchmarking that MEMSCOPE extends from single-core CPU benchmarking to heterogeneous memory on multicore systems.","marker":"[12]"},{"why":"Specifies the Zynq UltraScale+ MPSoC evaluation platform and the memory modules it contains.","marker":"[13]"},{"why":"Documents the ARM Cortex-A53 core, cache hierarchy, and performance-monitor registers that the implementation targets.","marker":"[14]"},{"why":"Provides the hypervisor-based cache partitioning used to create the private cache pool for the partition experiments.","marker":"[16]"},{"why":"Describes cache bank contention under hits, the effect MEMSCOPE reproduces and extends to an in-order Cortex-A53.","marker":"[17]"},{"why":"Documents the roughly 40 times bandwidth degradation from streaming writes that MEMSCOPE replicates with non-cacheable write streaming.","marker":"[19]"},{"why":"Supplies the vision benchmark suite used in the real-time application case study.","marker":"[20]"},{"why":"Provides the benchmark framework that maps application heaps to MEMSCOPE's exported user-space memory pools.","marker":"[21]"},{"why":"Is the established user-space bandwidth benchmark used to validate MEMSCOPE's cache-hit measurements.","marker":"[23]"}],"fun_headline_variants":["Kernel module maps memory latency under CPU-FPGA contention","MemScope: kernel-level benchmarks for heterogeneous memory","First kernel toolkit to pin down FPGA DRAM throttling","Benchmark suite dives into kernel to tame memory noise","How kernel control reveals 40x bandwidth loss under stress"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole measurement rests on the assumption that each device-tree mempool node names a real physical aperture that memremap maps directly and uncached, so the benchmark is actually touching that memory module and not a cached alias or a kernel-owned region.","fun_headline_variants_meta":{"raw":{"variants":["Kernel module maps memory latency under CPU-FPGA contention","MemScope: kernel-level benchmarks for heterogeneous memory","First kernel toolkit to pin down FPGA DRAM throttling","Benchmark suite dives into kernel to tame memory noise","How kernel control reveals 40x bandwidth loss under stress"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000239,"raw_usage":{"total_tokens":1480,"prompt_tokens":879,"completion_tokens":601,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":495,"completion_tokens_details":{"reasoning_tokens":521}},"tokens_in":495,"tokens_out":601,"duration_ms":6243,"temperature":1.0,"reasoning_tokens":521,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T04:31:28.304770+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"If a pool is declared with a device-tree node that points at an ordinary cached region rather than a dedicated uncached aperture, and MEMSCOPE still reports clean, low-variance results whose latency matches cache-hit latency instead of the memory module's true latency, then the memremap-based targeting is not actually measuring the intended memory.","supporting_citations":[{"cited_title":"nanobench: A low-overhead tool for running microbenchmarks on x86 systems,","cited_arxiv_id":null,"evidence_quote":"Establishes the precedent of kernel-level microbenchmarking that MEMSCOPE extends from single-core CPU benchmarking to heterogeneous memory on multicore systems."},{"cited_title":"Zynq ultrascale+ mpsoc data sheet: Overview (v1.8),","cited_arxiv_id":null,"evidence_quote":"Specifies the Zynq UltraScale+ MPSoC evaluation platform and the memory modules it contains."},{"cited_title":"Cortex-A53 MPCore technical reference manual (r0p4),","cited_arxiv_id":null,"evidence_quote":"Documents the ARM Cortex-A53 core, cache hierarchy, and performance-monitor registers that the implementation targets."},{"cited_title":"Jailhouse,","cited_arxiv_id":null,"evidence_quote":"Provides the hypervisor-based cache partitioning used to create the private cache pool for the partition experiments."},{"cited_title":"Cache bank-aware denial-of-service attacks on multicore arm processors,","cited_arxiv_id":null,"evidence_quote":"Describes cache bank contention under hits, the effect MEMSCOPE reproduces and extends to an in-order Cortex-A53."},{"cited_title":"Denial-of-Service Attacks on Shared Cache in Multicore: Analysis and Prevention ,","cited_arxiv_id":null,"evidence_quote":"Documents the roughly 40 times bandwidth degradation from streaming writes that MEMSCOPE replicates with non-cacheable write streaming."},{"cited_title":"SD-VBS: The san diego vision benchmark suite,","cited_arxiv_id":null,"evidence_quote":"Supplies the vision benchmark suite used in the real-time application case study."},{"cited_title":"Rt-bench: An extensible benchmark framework for the analysis and management of real-time applications,","cited_arxiv_id":null,"evidence_quote":"Provides the benchmark framework that maps application heaps to MEMSCOPE's exported user-space memory pools."},{"cited_title":"Taming non-blocking caches to improve isolation in multicore real-time systems,","cited_arxiv_id":null,"evidence_quote":"Is the established user-space bandwidth benchmark used to validate MEMSCOPE's cache-hit measurements."}],"review_version":1}