REVIEW 4 major objections 4 minor 29 references
Adaptive Migration Decision for Multi-Tenant Memory Systems
T0 review · 4 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read This paper claims that page migration in tiered memory systems is not always beneficial, and that a per-process, ping-pong-aware toggle between migration and no migration outperforms always-on migration.
desk verdict Worth a serious referee: a real CXL implementation of a sensible idea with a genuine blind spot for gradual working-set drift and a swapped-numbers bug. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the per-page ping-pong rate, tracked by a 'PagePromoted' flag: when a promoted page is later demoted, the kernel increments the vmstat counter demote_promoted for the owning process. A background thread computes the slope of this counter's two-second delta using central differences, and Algorithm 1 classifies the slope state as Varying, Stabilizing, or Stabilized against a threshold set to one quarter of the maximum observed slope. A second thread scans page-table access bits every five seconds with a two-megabyte stride to detect hot-set variation, and Algorithm 2 restarts migration when the accessed-PTE count diverges from a sliding-window mean. Per-process state stored in the task struct lets the framework toggle migration independently for each tenant, and a refault-distance heuristic decides promotion by comparing the intervals between consecutive hint faults for a page.
What would settle it
Run a synthetic workload whose hot set drifts on a time scale shorter than the two-second evaluation window, and check whether the framework keeps migrating through a no-benefit phase or stays stopped through a beneficial phase; if it does, the ping-pong metric and its timing thresholds are the point of failure.
Extended reading notes
Core claim
The central claim is that migration friendliness—whether repeated promotion and demotion cycles converge to a stable hot set in the fast tier—is a runtime-detectable property that determines whether page migration helps or hurts. The paper identifies the per-page ping-pong rate as the key signal: a high sustained rate of demoting pages that were just promoted indicates either a hot set larger than the fast tier or an effectively random access pattern, so further migration only adds fault-handling and copy overhead. A framework built on this signal stops migration for a process when the slope of the demote_promoted delta stabilizes at a high level, and restarts it when a sampled access-bit scan shows the accessed page count deviating from its recent mean. On a commercial CXL tiered-memory system, the authors report that this toggling yields performance comparable to the best hotness-based scheme when migration is useful and large gains when it is not.
Load-bearing premise
The framework's stop and restart decisions all rest on the assumption that the fixed two-second delta window, the one-quarter-of-max slope threshold, and the five-second access-bit scan with a two-megabyte stride correctly distinguish a hot set that is still moving from continuous useless migration across workloads and hardware.
Editorial extensions
If this is right
- On migration-unfriendly workloads, completely stopping migration can beat migrating, because the saved hint-fault handling and copy/demotion overhead outweigh any gain in fast-tier hit rate.
- Per-process migration control prevents one tenant's frequently but uniformly accessed pages from evicting another tenant's true hot set, so mixed friendly and unfriendly co-tenancy no longer degrades both.
- Migration can be restarted without hint-fault profiling; a sampled access-bit scan with a two-megabyte stride is enough to notice hot-set changes.
- Migration friendliness is not a fixed workload property: the same application can be unfriendly at small DRAM capacities and friendly at larger ones, so the runtime assessment must be made on the actual configuration.
- Because the implementation uses standard hint-fault machinery with only a small LRU bypass patch, the approach is portable to CPUs without hardware instruction sampling.
Reading between the lines
- The same ping-pong signal could be repurposed for other tiering decisions, such as whether to keep a page in high-bandwidth GPU memory or a disaggregated memory pool, where migration and copy costs are even higher.
- The fixed thresholds (two-second delta window, one-quarter-of-max slope, five-second scan, two-megabyte stride) are calibrated to the evaluated CXL prototype; workloads with sub-second hot-set changes may need an adaptive version of the algorithm.
- The reported multi-tenant gains are against a single strong non-exclusive migration baseline; a broader comparison against several per-process, sampling-based policies would isolate how much of the gain comes from friendliness-aware stopping versus the baseline LRU patch.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper argues that page migration in tiered memory systems is not always beneficial and that a migration-friendly-aware controller can improve performance by selectively stopping and restarting migration. It introduces two mechanisms: (1) a stop detector based on the delta of demote_promoted pages (ping-pong migrations) and its slope, evaluated by a background thread, and (2) a restart detector based on sampled page-table access bits, evaluated through a sliding-window mean of accessed PTE counts. The controller operates per process to handle multi-tenant interference. The implementation is on Linux v5.15 and is evaluated on a commercial CXL memory system with three DRAM sizes, comparing against No Migration, TPP-mod, MEMTIS, and NOMAD. The main reported results are 14.8% or 36.0% average improvement over NOMAD in single-tenant workloads and up to 72.0% in multi-tenant workloads.
Significance. If the results hold, the paper makes a useful contribution by reframing tiered-memory management from pure hotness-based migration to conditional migration, and by showing that per-process migration toggling can improve multi-tenant performance. The evaluation is broad: three DRAM capacities, single- and multi-tenant settings, real CXL hardware, and comparison against strong baselines. The observation that migration overhead can outweigh its benefit is well motivated and supported by the microarchitectural breakdown in Section 3.2. The paper also promises public source code, which would support reproducibility. However, the core claims depend on hand-tuned detection parameters and on the assumption that the decision algorithms correctly separate 'hot set still moving' from 'continuous useless migration'; the current evidence does not yet establish that generality.
major comments (4)
- [Section 4.3, Algorithm 2] Algorithm 2 line 6 contains a condition that does not implement the described logic: 'Count_accessed > (Mean_accessed >> 4)' is almost always true for any non-zero count of accessed PTEs, because Mean_accessed >> 4 is only a small fraction of the mean. As written, the restart counter would increase on nearly every iteration, making the detector useless. The surrounding text says the mechanism should detect a significant deviation from the average; the intended condition is presumably 'Count_accessed > Mean_accessed + (Mean_accessed >> 4)' or a two-sided deviation test. This must be corrected and the actual implemented condition should be stated precisely.
- [Sections 4.3 and 5.2] The restart detector uses a sliding-window mean, which absorbs gradual changes: if the hot set drifts slowly (e.g., shrinking from 50GB to 20GB over several minutes), each new accessed-PTE count stays close to the recent mean, the deviation never exceeds Mean>>4, and migration is never restarted. Algorithm 1 has the analogous problem for stopping: a slowly growing hot set yields a small slope of demote_promoted delta and is classified as Stabilizing. The microbenchmark in Section 5.2 validates only step-function phase changes, which are precisely the case where a level-based deviation detector can work. The paper therefore does not establish the central claim that the framework 'detects changes of memory access patterns' for gradual transitions. Please add an experiment with a continuously varying hot set and, if needed, modify the detector to use slope-based or change-point detection rather than level deviation.
- [Sections 4.2, 4.3, and 5] The stop/restart decisions depend on several hand-tuned constants: the 2s delta interval, the MaxSlope>>2 threshold, the 5s page-table scan interval, the 2MB scan stride, the Mean>>4 threshold, and an unspecified restart-counter threshold. No sensitivity analysis is provided for any of these values, and the evaluation does not vary them. Because the headline quantitative claims (14.8%, 36%, 72% improvements) rely on these decisions, the reader cannot tell whether the thresholds are workload-specific or robust. Please report a sensitivity sweep for at least the threshold factors and sampling intervals, and specify the restart threshold in Algorithm 2.
- [Section 5] The quantitative evaluation has no reported variance: single-tenant results appear to be single runs, and the '3 times' in Figure 11 refers to three different start-time offsets rather than repeated trials. In addition, the overhead of the two background threads (kevaluated and krestartd) and the page-table scans is not measured, despite the paper's repeated 'low cost' claim. Since the entire benefit of the scheme is that stopping migration avoids costs, the missing overhead measurement is load-bearing. Please report the number of runs and confidence intervals/standard deviations, and measure the CPU time, memory bandwidth, and TLB-shootdown overhead of the detection mechanisms.
minor comments (4)
- [Section 6 vs. Abstract] The conclusion swaps the two percentages: it says '14.8% performance improvement compared to NOMAD with migration-friendly workloads, while providing an average of 36.0% improvement with migration-unfriendly workloads,' but the abstract reports 14.8% for migration-unfriendly and 36.0% for migration-friendly. Please correct the inconsistency.
- [Table 1] The table uses symbols (O, triangle, X) but does not define what each symbol means in the caption or text; please add a legend.
- [Section 4.5] The refault-distance hot-page decision is presented as part of the design, but there is no ablation or isolated evaluation of this component; the paper should clarify how much of the reported improvement comes from the migration toggling versus the refault-distance promotion rule.
- [General] The paper would benefit from a description of the page-table scan implementation details: whether it walks only user-space VMAs, how it handles locked or non-present PTEs, and how the 2MB stride interacts with base pages.
Circularity Check
No circularity: empirical system paper with heuristic stop/restart mechanisms; no prediction reduces to a fit or to self-citation.
full rationale
The paper is a systems evaluation rather than a derived theoretical chain, and no claimed prediction reduces to a fitted parameter or to a self-citation by construction. Its central claim—that toggling migration based on per-process migration friendliness improves performance—is supported by direct benchmark measurements on a real CXL tiered-memory system against NOMAD, MEMTIS, and TPP-mod. The demote_promoted ping-pong metric is introduced as a runtime indicator with a mechanistic rationale (migration-unfriendly workloads repeatedly promote and demote pages, while friendly workloads eventually stabilize), not as a quantity defined in terms of the outcome it is used to predict. The stop and restart algorithms are heuristic controllers whose thresholds (2s delta interval, MaxSlope>>2, 5s access-bit scan, 2MB stride, Mean>>4 variation threshold) are engineering choices; there is no fitted value that is later renamed as a prediction, and the evaluation does not claim parameter-free derivation. The only self-citation by a coauthor, reference [9], appears in general surveys of hot-page detection approaches and does not carry any load-bearing argument. The restart mechanism's reliance on a sliding-window mean, and the microbenchmark's step-function phase changes, expose a possible limitation for gradual hot-set drift, but this is a correctness and sensitivity concern, not circular reasoning. No equation or design choice in the paper is equivalent to its inputs by construction, so the honest finding is no significant circularity.
Assumptions & free parameters
free parameters (6)
- Slope stabilization threshold factor =
MaxSlope >> 2 (one quarter of max observed slope)
- Delta sampling interval p =
2 seconds
- Page table scan interval =
5 seconds
- Scan stride =
2 MB
- Restart threshold in Algorithm 2 =
not specified numerically
- Refault distance promotion rule =
Promote if second distance < first distance
assumptions (4)
- domain assumption The demote_promoted metric, the count of demotions of previously promoted pages, reflects migration ping-pong and hence migration friendliness.
- domain assumption The Linux hint fault mechanism and page migration costs match the measurements in Section 3.2 (4-5us hint fault, 13-28us with migration).
- ad hoc to paper The evaluated benchmarks represent the population of real multi-tenant tiered memory workloads.
- domain assumption The CXL prototype (Samsung CMM-D) behaves like deployed CXL memory in latency and bandwidth.
Cite this review
Pith. "Pith review of Adaptive Migration Decision for Multi-Tenant Memory Systems." pith.science (2026). https://pith.science/paper/PPK2AQAY
@misc{pith2026250509164,
author = {Pith},
title = {Pith review of: Adaptive Migration Decision for Multi-Tenant Memory Systems},
year = {2026},
howpublished = {\url{https://pith.science/paper/PPK2AQAY}},
note = {Machine review of arXiv:2505.09164}
}
read the original abstract
Tiered memory systems consisting of fast small memory and slow large memory have emerged to provide high capacity memory in a cost-effective way. The effectiveness of tiered memory systems relies on how many memory accesses can be absorbed by the fast first-tier memory by page migration. The recent studies proposed several different ways of detecting hot pages and migrating them efficiently. However, our investigation shows that page migration is not always beneficial as it has the associated cost of detecting and migrating hot pages. When an application is unfriendly to migration, it is often better not to migrate pages at all. Based on the observation on migration friendliness, this paper proposes a migration control framework for multi-tenant tiered memory systems. First, it proposes a detection mechanism for migration friendliness, using per-page ping-pong status. Ping-pong pages which are promoted and demoted repeatedly in a short period of time tells migration effectiveness. Based on their change behaviors, migration is stopped or continued. After the page migration is stopped, the second mechanism detects changes of memory access patterns in a low cost way to determine whether migration needs to be resumed. Finally, as each application has a different behavior, our framework provides per-process migration control to selectively stop and start migration depending on application characteristics. We implement the framework in the Linux kernel. The evaluation with a commercial CXL-based tiered memory system shows that it effectively controls migration in single and multi-tenant environments.
Figures
Figures from the paper (8 more)
Reference graph
Works this paper leans on
-
[1]
Emmanuel Amaro, Christopher Branner-Augmon, Zhi- hongLuo,AmyOusterhout,MarcosKAguilera,Aurojit Panda, Sylvia Ratnasamy, and Scott Shenker. Can far memory improve job throughput? InProceedings of the 12 Fifteenth European Conference on Computer Systems, pages 1–16, 2020
work page 2020
-
[2]
Reconsidering os memory opti- mizations in the presence of disaggregated memory
ShaiBergman,PriyankFaldu,BorisGrot,LluísVilanova, and Mark Silberstein. Reconsidering os memory opti- mizations in the presence of disaggregated memory. In Proceedings of the 2022 ACM SIGPLAN International Symposium on Memory Management, pages 1–14, 2022
work page 2022
-
[3]
Facebook and amazon are causing a memory shortage
Brandon Butler. Facebook and amazon are causing a memory shortage. Network World, 2012
work page 2012
- [4]
-
[5]
Incorporating instruction-based sampling into amd codeanalyst
Paul Drongowski, Lei Yu, Frank Swehosky, Suravee Suthikulpanit, and Robert Richter. Incorporating instruction-based sampling into amd codeanalyst. In 2010 IEEE International Symposium on Performance Analysis of Systems & Software (ISPASS),pages119–120. IEEE, 2010
work page 2010
-
[6]
Data tiering in het- erogeneous memory systems
Subramanya R Dulloor, Amitabha Roy, Zheguang Zhao, NarayananSundaram,NadathurSatish,RajeshSankaran, Jeff Jackson, and Karsten Schwan. Data tiering in het- erogeneous memory systems. In Proceedings of the Eleventh European Conference on Computer Systems, pages 1–16, 2016
work page 2016
-
[7]
Efficient memory dis- aggregation with infiniswap
JunchengGu,YoungmoonLee,YiwenZhang,Mosharaf Chowdhury, and Kang G Shin. Efficient memory dis- aggregation with infiniswap. In14th USENIX Sympo- sium on Networked Systems Design and Implementation (NSDI 17), pages 649–667, 2017
work page 2017
-
[8]
Hetero- visor: Exploiting resource heterogeneity to enhance the elasticity of cloud platforms.ACM SIGPLAN Notices, 50(7):79–92, 2015
Vishal Gupta, Min Lee, and Karsten Schwan. Hetero- visor: Exploiting resource heterogeneity to enhance the elasticity of cloud platforms.ACM SIGPLAN Notices, 50(7):79–92, 2015
2015
Show all 29 references
-
[9]
Adaptivepagemigrationpolicywithhuge pages in tiered memory systems.IEEE Transactions on Computers, 71(1):53–68, 2020
Taekyung Heo, Yang Wang, Wei Cui, Jaehyuk Huh, and LintaoZhang. Adaptivepagemigrationpolicywithhuge pages in tiered memory systems.IEEE Transactions on Computers, 71(1):53–68, 2020
2020
-
[10]
Autotm:Automatic tensor movement in heterogeneous memory systems usingintegerlinearprogramming
Mark Hildebrand, Jawad Khan, Sanjeev Trika, Jason Lowe-Power,andVenkateshAkella. Autotm:Automatic tensor movement in heterogeneous memory systems usingintegerlinearprogramming. In Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming...
2020
-
[11]
Intel 64 and IA-32 Architectures Software Developer’s Manual, Volume 3B: System Pro- gramming Guide, 2023
Intel Corporation. Intel 64 and IA-32 Architectures Software Developer’s Manual, Volume 3B: System Pro- gramming Guide, 2023
2023
-
[12]
Basic per- formancemeasurementsoftheinteloptanedcpersistent memory module
JosephIzraelevitz,JianYang,LuZhang,JunoKim,Xiao Liu, Amirsaman Memaripour, Yun Joon Soh, Zixuan Wang, Yi Xu, Subramanya R Dulloor, et al. Basic per- formancemeasurementsoftheinteloptanedcpersistent memory module. arXiv preprint arXiv:1903.05714 , 2019
1903 arXiv
-
[13]
Demystifyingacxl type-2 device: A heterogeneous cooperative computing perspective
HouxiangJi,SrikarVanavasam,YangZhou,QirongXia, Jinghan Huang, Yifan Yuan, Ren Wang, Pekon Gupta, BhushanChitlur,IpoomJeong,etal. Demystifyingacxl type-2 device: A heterogeneous cooperative computing perspective. In 2024 57th IEEE/ACM International Symposium on Microarchitectur...
2024
-
[14]
Hbm (high bandwidth memory) dram technology and architecture
HongshinJun,JinheeCho,KangseolLee,Ho-YoungSon, Kwiwook Kim, Hanho Jin, and Keith Kim. Hbm (high bandwidth memory) dram technology and architecture. In 2017 IEEE International Memory Workshop (IMW), pages 1–4. IEEE, 2017
2017
-
[15]
Heteroos—osdesignforheterogeneous memory management in datacenter
Sudarsun Kannan, Ada Gavrilovska, Vishal Gupta, and KarstenSchwan. Heteroos—osdesignforheterogeneous memory management in datacenter. in 2017 acm/ieee 44thannualinternationalsymposiumoncomputerarchi- tecture (isca). IEEE: Piscataway, NJ, USA, 2017
2017
-
[16]
Memtis: Efficient memory tiering with dynamic page classification and page size deter- mination
Taehyung Lee, Sumit Kumar Monga, Changwoo Min, and Young Ik Eom. Memtis: Efficient memory tiering with dynamic page classification and page size deter- mination. In Proceedings of the 29th Symposium on Operating Systems Principles, pages 17–34, 2023
2023
-
[17]
Multi- clock: Dynamic tiering for hybrid memory systems
Adnan Maruf, Ashikee Ghosh, Janki Bhimani, Daniel Campello, Andy Rudoff, and Raju Rangaswami. Multi- clock: Dynamic tiering for hybrid memory systems. In 2022 IEEE International Symposium on High- Performance Computer Architecture (HPCA ’22), 2022
2022
-
[18]
Tpp:Transparentpageplacement for cxl-enabled tiered-memory
Hasan Al Maruf, Hao Wang, Abhishek Dhanotia, Jo- hannes Weiner, Niket Agarwal, Pallab Bhattacharya, ChrisPetersen,MosharafChowdhury,ShobhitKanaujia, andPrakashChauhan. Tpp:Transparentpageplacement for cxl-enabled tiered-memory. InProceedings of the 28th ACM International Confe...
2023
-
[19]
Maphea: A lightweight memory hierarchy-aware profile-guided heap allocation framework
Deok-Jae Oh, Yaebin Moon, Eojin Lee, Tae Jun Ham, Yongjun Park, Jae W Lee, and Jung Ho Ahn. Maphea: A lightweight memory hierarchy-aware profile-guided heap allocation framework. InProceedings of the 22nd ACM SIGPLAN/SIGBED International Conference on Languages, Compilers, and...
2021
-
[20]
Chatgpt: Large language model
OpenAI. Chatgpt: Large language model. https: //chat.openai.com, 2023. 13
2023
-
[21]
Thecaseforramclouds:scalablehigh- performance storage entirely in dram.ACM SIGOPS Operating Systems Review, 43(4):92–105, 2010
JohnOusterhout,ParagAgrawal,DavidErickson,Chris- tos Kozyrakis, Jacob Leverich, David Mazières, Subha- sish Mitra, Aravind Narayanan, Guru Parulkar, Mendel Rosenblum,etal. Thecaseforramclouds:scalablehigh- performance storage entirely in dram.ACM SIGOPS Operating Systems Revie...
2010
-
[22]
Patch submitted to linux-mm mailing list
Bharata B Rao. Patch submitted to linux-mm mailing list. https://patchwork.kernel.org/ project/linux-mm/patch/20240327160237. 2355-2-bharata@amd.com/, 2024. Accessed: 2025-01-08
2024
-
[23]
Lightweight frequency-based tiering for cxl memory systems.arXiv preprint arXiv:2312.04789, 2023
Kevin Song, Jiacheng Yang, Sihang Liu, and Gennady Pekhimenko. Lightweight frequency-based tiering for cxl memory systems.arXiv preprint arXiv:2312.04789, 2023
2023 arXiv
-
[24]
Demystifyingcxlmemory with genuine cxl-ready systems and devices
Yan Sun, Yifan Yuan, Zeduo Yu, Reese Kuper, Chihun Song, Jinghan Huang, Houxiang Ji, Siddharth Agarwal, JiaqiLou,IpoomJeong,etal. Demystifyingcxlmemory with genuine cxl-ready systems and devices. InPro- ceedings of the 56th Annual IEEE/ACM International Symposium on Microarchi...
2023
-
[25]
Llama: Open and efficient foundation language models
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet,Marie-AnneLachaux,TimothéeLacroix,Bap- tiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023
2023 arXiv
-
[26]
Automatic numa bal- ancing
Rik van Riel and Vinod Chegu. Automatic numa bal- ancing. Red Hat Summit, 2014
2014
-
[27]
Nomad:non-exclusive memory tiering via transactional page migration
Lingfeng Xiang, Zhen Lin, Weishu Deng, Hui Lu, Jia Rao, Yifan Yuan, and Ren Wang. Nomad:non-exclusive memory tiering via transactional page migration. In 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI 24), pages 19–35, 2024
2024
-
[28]
Flexmem: Adaptive page profiling and mi- gration for tiered memory
Dong Xu, Junhee Ryu, Kwangsik Shin, Pengfei Su, and Dong Li. Flexmem: Adaptive page profiling and mi- gration for tiered memory. In2024 USENIX Annual Technical Conference (USENIX ATC 24),pages817–833, 2024
2024
-
[29]
Nimble page management for tiered memory systems
Zi Yan, Daniel Lustig, David Nellans, and Abhishek Bhattacharjee. Nimble page management for tiered memory systems. InProceedings of the Twenty-Fourth International Conference on Architectural Support for Programming Languages and Operating Systems, pages 331–345, 2019. 14
2019
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.