Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

Managed-Retention Memory: A New Class of Memory for the AI Era

T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A proposed new memory class, Managed-Retention Memory, would relax persistence from years to hours so that storage-class memory technologies can outperform DRAM and HBM on read throughput, energy, and capacity for AI inference.

desk verdict A clear, honest position paper proposing a new memory class for AI inference; the direction is plausible but the endurance analysis rests on an unvalidated average-writes assumption. read the letter →

arxiv 2501.09605 v1 pith:62DZOI6V submitted 2025-01-16 cs.AR cs.AIcs.DCcs.ET

Sergey Legtchenko , Ioan Stefanovici , Richard Black , Antony Rowstron , Junyi Liu , Paolo Costa , Burcu Canakci , Dushyanth Narayanan
show 1 more author
Xingbo Wu
This is my paper · ORCID
classification cs.ARcs.AIcs.DCcs.ET
keywords Managed-RetentionMemorystorageclassAIinferenceHBMKVcacheenduranceretentiontimeemergingtechnologies
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that AI inference workloads are so read-dominated and so tolerant of soft state that long-term persistence is the wrong thing to optimize. It proposes Managed-Retention Memory (MRM), a memory class that keeps data for hours or days instead of ten-plus years, and spends the resulting savings on endurance, density, and energy efficiency. If MRM works, technologies like PCM, RRAM, and STT-MRAM that failed as storage-class memory could become viable in AI accelerators, reducing dependence on expensive, yield-limited HBM. The argument rests on workload numbers: each generated token re-reads the weights and the whole KV cache, giving read:write ratios above 1000:1, while the data is either persisted elsewhere or recomputable.

What carries the argument

The central mechanism is the retention–endurance–energy trade-off, applied through a workload characterization of autoregressive inference: weights and KV caches are read sequentially and repeatedly for every generated token, writes are mostly appends, and the data is recoverable or soft. The quantitative engine is a five-year endurance requirement computed from reported inference throughputs and context lengths, compared against device endurance of Intel Optane PCM, Weebit RRAM, and Everspin STT-MRAM as well as the potential endurance of these technologies. That comparison identifies the gap that MRM would fill by right-provisioning retention time to the workload.

What would settle it

Run a long-term measurement of cell-level write locality on a production LLM serving workload or a simulation with a real KV-cache allocator: if any cell or block exceeds the stated endurance before the five-year target, the central MRM viability claim fails; a prototype device with hour-scale retention that sustains the read bandwidth and energy targets would confirm it.

Watch

Extended reading notes

Core claim

The central claim is that the conventional non-volatile/volatile distinction is the wrong design constraint for AI inference memory. The three dominant data structures in inference are model weights (effectively immutable in memory), KV caches (soft state that can be recomputed), and activations (transient), none of which need decade-scale persistence; what they need is high sequential read bandwidth, density, and low energy per bit. Because existing SCM devices spend retention on ten-plus-year persistence, they underdeliver on endurance and speed, and the paper argues that re-targeting them at day- or hour-scale retention, as MRM does, lets the underlying technologies meet AI inference needs. Its quantitative support is a five-year endurance budget: for hourly model updates and KV-cache writes at Llama2-70B throughputs, the average writes per cell sit below the potential endurance of PCM, RRAM, and STT-MRAM, while HBM is vastly overprovisioned on endurance.

Load-bearing premise

The viability calculation assumes writes are spread evenly enough across memory that the average number of writes per cell is the right endurance target, but KV-cache appends and weight updates could concentrate on particular cells.

Editorial extensions

If this is right

  • AI accelerators could use a cheaper, denser memory tier for weights and KV caches, leaving HBM for write-heavy structures like activations.
  • Device vendors would have a concrete target: redesign PCM, RRAM, or STT-MRAM for hour-scale retention rather than decade-scale persistence.
  • Memory controllers could become simpler block-level sequential devices, with refresh, wear-leveling, and retention tracking pushed into cluster-level software.
  • Systems software would need retention-aware data placement and refresh scheduling, and possibly dynamically programmable retention times.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is to measure real cell-level write locality in LLM serving; if the peak-to-average write ratio is high, the average-writes-per-cell calculation would need revision, though software-managed refresh could mitigate hot cells.
  • The same retention-relaxation logic likely applies to other read-dominated soft-state workloads, such as embedding lookups or activation offloading, not only LLM inference.
  • If dynamic retention control is feasible, retention time becomes a schedulable resource, opening the way to memory controllers and OS abstractions that treat retention as a tunable quality of service.
  • The decisive experiment is a prototype PCM or RRAM array with hour-scale retention measured against HBM for read throughput, energy, and endurance; until then the viability claim rests on device-physics scaling assumptions.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes a new memory class called Managed-Retention Memory (MRM), intended for AI inference workloads. The central idea is to relax the retention time of storage-class-memory technologies (PCM, RRAM, STT-MRAM) from the traditional 10+ years to hours or days, and in return obtain better endurance, read throughput, energy efficiency, and density. The paper characterizes foundation-model inference as read-dominated, sequential, and concentrated on model weights and the KV cache; argues that HBM is overprovisioned for this workload on write performance while underprovisioned on density and read bandwidth; estimates endurance requirements using average writes per cell; and sketches software-stack implications including retention-aware data placement, lightweight block-level memory controllers, dynamically configurable retention, and retention-aware error correction. The paper is explicitly framed as a call for action rather than a completed implementation.

Significance. If the central claims hold, MRM would be a significant memory-class proposal with the potential to reduce AI-infrastructure cost and energy while reviving SCM technologies that have failed to displace DRAM. The paper has real strengths: it identifies a plausible and important trade-off space, gives a clear workload characterization of LLM inference, connects to a broad body of prior work on retention/endurance trade-offs, and makes a falsifiable prediction that relaxed retention can make PCM/RRAM/STT-MRAM viable for AI inference memory. However, the paper currently provides only one quantitative figure as supporting evidence, and that figure rests on an averaging assumption that is not justified. The abstract's claim about HBM suboptimality is also asserted rather than demonstrated. These gaps are load-bearing for the proposal's credibility.

major comments (3)
  1. [Section 3, Figure 1] The endurance-requirement calculation is load-bearing and currently rests on an averaging assumption that is not justified. The text states 'we compute the number of KV cache writes, and infer the average number of writes per cell,' but KV-cache appends are allocated per context and weight updates are bulk overwrites, so per-cell write counts depend on allocation and free order and can have a heavy tail, for example when pages are reused frequently under PagedAttention-style allocation. Figure 1 plots this average against device and 'potential' endurance, yet Section 4's proposed software wear-leveling is described only as a design direction ('wear-leveling ... can be left up to a software control plane') with no algorithm, uniformity guarantee, or overhead analysis. If peak cell wear is the binding constraint, the plotted device endurance values may be insufficient even for the stated workload, and the viability of MRM is not established. Please provide either a peak-wear analysis for concrete allocation policies or a demonstrated wear-leveling scheme with uniformity guarantees.
  2. [Abstract and Section 2.2] The motivating claim that HBM is 'overprovisioned on write performance, but underprovisioned on density and read bandwidth' is stated as 'Analysis shows' in the abstract, but no quantitative analysis appears in Section 2.2 or elsewhere. The section describes workload properties qualitatively, noting read:write ratios over 1000:1 and sequential access, and it discusses HBM scaling, yield, and cost, but it does not provide the memory-bandwidth, energy-per-bit, or density numbers needed to establish the mismatch. Because this claim motivates the entire MRM proposal, please add concrete comparisons, such as measured or cited read/write bandwidth and energy per bit for HBM versus the workload's demands, or explicitly reframe the claim as a hypothesis to be tested.
  3. [Section 3] The central promise that MRM 'can potentially enable' PCM, RRAM, and STT-MRAM to outperform DRAM and HBM 'on the key metrics such as read throughput, energy efficiency and capacity' is supported only by a qualitative citation to [16] and by references to device trade-offs. No quantitative projections are given for read throughput, energy per bit, or density as a function of retention time, so the reader cannot assess whether relaxing retention from 10+ years to hours or days yields improvements of the required magnitude. Please include a quantitative model or sensitivity analysis using published device parameters, or state explicitly that these performance gains are assumptions to be evaluated in future work.
minor comments (4)
  1. [Throughout] There are several typos and informal spellings that should be corrected: 'firsttrained' (Section 2), 'reasonning' (Section 2), 'hollistic' (Section 4), 'strigent' (Section 4), 'tighly' (Section 5), and 'Transfmer' in reference [20].
  2. [References] Reference [43] has a malformed URL ending in '.pdfl'; please correct the link.
  3. [Figure 1] Figure 1 is not fully described: the axes, units, and data sources for each bar are not specified in the caption, and the distinction between 'device' and 'potential' endurance is not defined in the text.
  4. [Section 4] The paper proposes a block-level memory controller and software-managed wear-leveling, but it does not explain how these interact with the KV cache's append pattern and page-granular allocation; a brief discussion of this interaction would improve the systems argument.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper argues from external workload data and device specifications; no derivation reduces to its own inputs.

full rationale

The paper is a position/architecture proposal rather than a quantitative derivation. The load-bearing estimate in Figure 1 compares workload endurance requirements to SCM device and potential endurance values. The requirement side is computed from externally reported inference throughputs and median context lengths from Splitwise, combined with an explicit average per-cell write calculation; the device side is taken from published device specifications and independent technology surveys. Neither side is fitted to the paper's own conclusion, and the MRM proposal does not use same-author citations or a uniqueness theorem to force its choice. The most fragile point, the use of average writes per cell despite possible locality of KV-cache and weight overwrites, is a correctness/robustness concern rather than a circular one: the estimate could be wrong without being definitionally equal to the claimed outcome. Since no equation-level self-reference, fitted-parameter-as-prediction, or load-bearing self-citation is present, the correct finding is no significant circularity.

Assumptions & free parameters 2 free parameters · 4 assumptions · 1 invented entities

The central claim rests on several unverified assumptions: the read-dominated nature of inference, the lack of need for persistence, the direction and magnitude of the retention trade-off, and the validity of the average-writes-per-cell estimate. These are domain assumptions or ad-hoc modeling choices rather than derived facts, which is appropriate for a position paper but limits the strength of the conclusion.

free parameters (2)
  • Weight update frequency scenarios = 1/hour and 1/second
    Hand-picked scenarios in Section 3 bracket the endurance required for model weight writes; the per-second case is called 'intensive' but no workload evidence is given for it.
  • KV cache write distribution = average writes per cell
    The endurance requirement is computed as total KV cache writes divided by cells, assuming uniform wear; actual write locality could raise per-cell requirements significantly.
assumptions (4)
  • domain assumption Foundation model inference is read-dominated, sequential, and predictable, with read:write ratios over 1000:1.
    Stated in Section 2.2 based on current autoregressive transformer workloads. If future inference changes (e.g., more frequent weight updates, different architectures), the premise weakens.
  • domain assumption Weights and KV caches do not require long-term persistence; weights are durable elsewhere and KV cache is soft state.
    The basis for relaxing retention to hours or days, stated in Sections 2 and 3. If deployment requires surviving power loss or reboot without recomputation, MRM would need additional refresh mechanisms.
  • ad hoc to paper Relaxing retention time will improve read throughput, energy efficiency, and/or density in PCM, RRAM, and STT-MRAM.
    The paper relies on trade-off references [18, 31] and 'potential' numbers [27, 46] but does not demonstrate the magnitude of these gains for MRM-style operation. The entire viability argument rests on this assumed trade-off.
  • ad hoc to paper Average writes per cell is a valid estimate of the endurance requirement.
    Figure 1 uses this average as the workload endurance requirement. Wear concentration in KV cache append regions could invalidate the estimate and change the figure's conclusion.
invented entities (1)
  • Managed-Retention Memory (MRM)
    purpose: Proposed new memory class with hours-to-days retention that trades long-term persistence for higher read throughput, density, energy efficiency, and endurance in AI inference.
    No device, prototype, or measurement is presented; the paper explicitly frames MRM as an opportunity space and call for action. It has no falsifiable handle outside the paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Managed-Retention Memory: A New Class of Memory for the AI Era." pith.science (2026). https://pith.science/paper/62DZOI6V

@misc{pith2026250109605,
  author       = {Pith},
  title        = {Pith review of: Managed-Retention Memory: A New Class of Memory for the AI Era},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/62DZOI6V}},
  note         = {Machine review of arXiv:2501.09605}
}
read the original abstract

AI clusters today are one of the major uses of High Bandwidth Memory (HBM). However, HBM is suboptimal for AI workloads for several reasons. Analysis shows HBM is overprovisioned on write performance, but underprovisioned on density and read bandwidth, and also has significant energy per bit overheads. It is also expensive, with lower yield than DRAM due to manufacturing complexity. We propose a new memory class: Managed-Retention Memory (MRM), which is more optimized to store key data structures for AI inference workloads. We believe that MRM may finally provide a path to viability for technologies that were originally proposed to support Storage Class Memory (SCM). These technologies traditionally offered long-term persistence (10+ years) but provided poor IO performance and/or endurance. MRM makes different trade-offs, and by understanding the workload IO patterns, MRM foregoes long-term data retention and write performance for better potential performance on the metrics important for these workloads.

Figures

Figures reproduced from arXiv: 2501.09605 by the authors.

Figure 1
Figure 1. Endurance requirements for KV cache and model weights vs. endurance of memory technologies. 3 THE MEMORY OPPORTUNITY We posit that the combination of (i) the importance and scale of foundation model infrastructure, (ii) the large difference between the workload patterns of conventional server CPUs and that of AI accelerators, and (iii) the poor match of HBM to the workload, opens a field of computer architecture res… view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. DTCO of NOR-Type IGZO FeFETs for 3D Heterogeneous AI Memories: A Read-Centric Perspective

    cs.ET 2026-04 unverdicted novelty 5.5 of 10

    Read-centric DTCO shows NOR IGZO FeFETs can hit 10-A SRAM-equivalent density and sub-5 ns access for AI memory tiers, but need positive-Vt engineering and face 3D stacking limits from sneak current.

Reference graph

Works this paper leans on

58 extracted references · 43 canonical work pages · cited by 1 Pith paper

  1. [16]

    Engin Ipek, Jeremy Condit, Edmund B Nightingale, Doug Burger, and Thomas Moscibroda. 2010. Dynamically Replicated Memory: Building Reliable Systems from Nanoscale Resistive Memo- ries. In ASPLOS 2010: 15th International Conference on Archi- tectural Support for Programming Languages and Operating Systems, Pittsburgh, PA . ACM. https://www.microsoft.com/en...

    work page 2010

  2. [1]

    Next-generation memory for computers

    2025. Next-generation memory for computers. https://www. intrinsicsemi.com/

    work page 2025

  3. [2]

    The ReRAM Market Opportunity

    2025. The ReRAM Market Opportunity. https://www.weebit-nano. com/market/market-overview/

    work page 2025

  4. [3]

    Gulavani, and Ramachandran Ramjee

    Amey Agrawal, Ashish Panwar, Jayashree Mohan, Nipun Kwatra, Bhargav S. Gulavani, and Ramachandran Ramjee. 2023. SARATHI: Ef- ficient LLM Inference by Piggybacking Decodes with Chunked Prefills. arXiv:2308.16369 [cs.LG] https://arxiv.org/abs/2308.16369

    arXiv 2023

  5. [4]

    Transformer inference tricks

    Artificial-Fintelligence 2023. Transformer inference tricks. https: //www.artfintel.com/p/transformer-inference-tricks

    work page 2023

  6. [5]

    Is Optane DIMM endurance good enough? Quick answer

    blocksandfiles.com 2019. Is Optane DIMM endurance good enough? Quick answer. . . Yes, Intel has delivered. https://blocksandfiles.com/2019/04/04/enduring-optane-dimm- question-is-its-endurance-good-enough-yes-intel-has-delivered/

    work page 2019

  7. [6]

    CrossBar tries to secure embedded ReRAM IoT market

    blocksandfiles.com 2022. CrossBar tries to secure embedded ReRAM IoT market. https://blocksandfiles.com/2022/04/21/no-sniffing- crossbar-tries-to-secure-its-embedded-reram-iot-market-niche/

    work page 2022

  8. [7]

    Yuan-Hao Chang, Jen-Wei Hsieh, and Tei-Wei Kuo. 2007. Endurance enhancement of flash-memory storage systems: An efficient static wear leveling design. In Proceedings of the 44th annual Design Automa- tion Conference. 212–217

    work page 2007

Show all 58 references
  1. [8]

    Dolinar, Dariush Divsalar, and F

    S. Dolinar, Dariush Divsalar, and F. Pollara. 1998. Code Performance as a Function of Block Size. Telecommunications and Mission Operations Progress Report (01 1998)

    1998

  2. [9]

    Marcello Duhalde, Alain Greiner, and Frederic Petrot. 1995. A high performance modular embedded ROM architecture. In 1995 IEEE In- ternational Symposium on Circuits and Systems (ISCAS) , Vol. 2. IEEE, 1057–1060

    1995

  3. [10]

    D. Fick. 2022. Analog Compute-in-Memory For AI Edge Inference. In 2022 International Electron Devices Meeting (IEDM) . 21.8.1–21.8.4. https://doi.org/10.1109/IEDM45625.2022.10019367

    2022

  4. [11]

    Yasmin Halawani, Baker Mohammad, and Hani Saleh. 2021. Design Exploration of ReRAM-Based Crossbar for AI Inference. IEEE Access 9 (2021), 70430–70442. https://doi.org/10.1109/ACCESS.2021.3076445

    2021

  5. [12]

    Loh, and Doug Burger

    Andrew Hay, Karin Strauss, Timothy Sherwood, Gabriel H. Loh, and Doug Burger. 2011. Preventing PCM banks from seizing too much power. In 2011 44th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). 186–195

    2011

  6. [13]

    W. T. Huang, C. T. Chen, C. H. Chen, and C. C. Cheng. 2008. Energy- Efficient Buffer Architecture for Flash Memory. In 2008 International Conference on Multimedia and Ubiquitous Engineering (mue 2008) . 543–

    2008

  7. [14]

    Ielmini, F

    D. Ielmini, F. Nardi, C. Cagli, and A. L. Lacaita. 2010. Trade-off between data retention and reset in NiO RRAMS. In 2010 IEEE International Reliability Physics Symposium . 620–626. https://doi.org/10.1109/IRPS. 2010.5488761

    2010

  8. [15]

    Intel Optane Memory - Responsive Memory, Accelerated Performance

    Intel 2019. Intel Optane Memory - Responsive Memory, Accelerated Performance. https://www.intel.com/content/www/us/en/products/ details/memory-storage/optane-memory.html

    2019

  9. [17]

    Mishra, Cong Xu, Yuan Xie, Vijaykrishnan Narayanan, Ravishankar Iyer, and Chita R

    Adwait Jog, Asit K. Mishra, Cong Xu, Yuan Xie, Vijaykrishnan Narayanan, Ravishankar Iyer, and Chita R. Das. 2012. Cache re- vive: Architecting volatile STT-RAM caches for enhanced perfor- mance in CMPs. In DAC Design Automation Conference 2012. 243–252. https://doi.org/10.1145...

    2012

  10. [18]

    Myoungsoo Jung, Youngbin Jin, and Mustafa Shihab. 2014. Area, Power and Latency Considerations of STT-MRAM to Substitute for Main Memory

    2014

  11. [19]

    Byeongho Kim, Sanghoon Cha, Sangsoo Park, Jieun Lee, Sukhan Lee, Shin-haeng Kang, Jinin So, Kyungsoo Kim, Jin Jung, Jong-Geon Lee, Sunjung Lee, Yoonah Paik, Hyeonsu Kim, Jin-Seong Kim, Won-Jo Lee, Yuhwan Ro, YeonGon Cho, Jin Hyun Kim, JoonHo Song, Jaehoon Yu, Seungwon Lee, Jeo...

    2024

  12. [20]

    Jin Hyun Kim, Yuhwan Ro, Jinin So, Sukhan Lee, Shin-haeng Kang, YeonGon Cho, Hyeonsu Kim, Byeongho Kim, Kyungsoo Kim, Sangsoo Park, Jin-Seong Kim, Sanghoon Cha, Won-Jo Lee, Jin Jung, Jong-Geon Lee, Jieun Lee, JoonHo Song, Seungwon Lee, Jeonghyeon Cho, Jaehoon Yu, and Kyomin So...

    2023

  13. [21]

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Sto- ica. 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention. In Proceedings of the 29th Symposium on Operating Systems Princ...

    2023

  14. [22]

    Corey Lammie, Mostafa Rahimi Azghadi, and Daniele Ielmini. 2021. Empirical metal-oxide RRAM device endurance and retention model for deep learning simulations. Semiconductor Science and Technology 36, 6 (apr 2021), 065003. https://doi.org/10.1088/1361-6641/abf29d

    2021 doi

  15. [23]

    Lee, Engin Ipek, Onur Mutlu, and Doug Burger

    Benjamin C. Lee, Engin Ipek, Onur Mutlu, and Doug Burger. 2009. Architecting phase change memory as a scalable dram alternative. SIGARCH Comput. Archit. News 37, 3 (June 2009), 2–13. https://doi. org/10.1145/1555815.1555758

    2009

  16. [24]

    Philip Wong

    Shuhan Liu, Shengjun Qin, Koustav Jana, Jian Chen, Kasidit Toprasert- pong, and H.-S. Philip Wong. 2024. First Experimental Demon- stration of Hybrid Gain Cell Memory with Si PMOS and ITO FET for High-speed On-chip Memory. In 2024 IEEE Symposium on VLSI Technology and Circuits...

    2024

  17. [25]

    Yuhan Liu, Hanchen Li, Yihua Cheng, Siddhant Ray, Yuyang Huang, Qizheng Zhang, Kuntai Du, Jiayi Yao, Shan Lu, Ganesh Anantha- narayanan, Michael Maire, Henry Hoffmann, Ari Holtzman, and Junchen Jiang. 2024. CacheGen: KV Cache Compression and Stream- ing for Fast Large Language...

    2024

  18. [26]

    Fujio Masuoka, Masaki Momodomi, Yoshihisa Iwata, and Riichiro Shirota. 1987. New ultra high density EPROM and flash EEPROM with NAND structure cell. In 1987 International Electron Devices Meeting . IEEE, 552–555

    1987

  19. [27]

    Sze, Umesh Chand, and Tseung Yuen Tseng

    Jagan Singh Meena, Simon M. Sze, Umesh Chand, and Tseung Yuen Tseng. 2014. Overview of emerging nonvolatile memory technolo- gies. Nanoscale Research Letters 9 (2014), 526 – 526. https://api. semanticscholar.org/CorpusID:3932089 6 Managed-Retention Memory: A New Class of Memor...

    2014

  20. [28]

    Xupeng Miao, Gabriele Oliaro, Zhihao Zhang, Xinhao Cheng, Zeyu Wang, Zhengxin Zhang, Rae Ying Yee Wong, Alan Zhu, Lijie Yang, Xiaoxiang Shi, et al. 2023. SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification. arX...

    2023 arXiv

  21. [29]

    Gabriel Molas, Giuseppe Piccolboni, Alessandro Bricalli, Anthonin Verdy, I. Naot, Y. Cohen, Amir Regev, Ishai Naveh, Damien Deleruyelle, Quentin Rafhay, Niccolo Castellani, Lucas Reganaz, Alain Persico, R. Segaud, Jean-François Nodin, Valentina Meli, Shelia A. Martin, François...

    2022

  22. [30]

    Carlo Montangero. 1974. An approach to the optimal specification of read-only memories in microprogrammed digital computers. IEEE Trans. Comput. 100, 4 (1974), 375–389

    1974

  23. [31]

    C. Nail, G. Molas, Philippe Blaise, Giuseppe Piccolboni, Benoit Sklé- nard, Carlo Cagli, M. Bernard, Anne Roule, Muhamad Azzaz, E. Vianello, C. Carabasse, R. Berthier, David Cooper, C. Pelissier, T. Magis, Gerard Ghibaudo, Christophe Vallée, D. Bedeau, O. Mosendz, and L. Perni...

    2016

  24. [32]

    The NVIDIA Blackwell Architecture

    nvidia.com 2024. The NVIDIA Blackwell Architecture. https:// resources.nvidia.com/en-us-blackwell-architecture?ncid=no-ncid

    2024

  25. [33]

    Veera Papirla and Chaitali Chakrabarti. 2009. Energy-aware error control coding for flash memories. In Proceedings of the 46th Annual Design Automation Conference. 658–663

    2009

  26. [35]

    Pratyush Patel, Esha Choukse, Chaojie Zhang, Aashaka Shah, Íñigo Goiri, Saeed Maleki, and Ricardo Bianchini. 2024. Split- wise: Efficient generative LLM inference using phase splitting. In ISCA. https://www.microsoft.com/en-us/research/publication/ splitwise-efficient-generati...

    2024

  27. [36]

    Microsoft plans to invest $80 billion on AI-enabled data centers in fiscal 2025

    Reuters.com 2025. Microsoft plans to invest $80 billion on AI-enabled data centers in fiscal 2025. https://www.reuters.com/technology/ artificial-intelligence/microsoft-plans-spend-80-bln-ai-enabled- data-centers-fiscal-2025-cnbc-reports-2025-01-03/

    2025

  28. [37]

    D. Shum, D. Houssameddine, S. T. Woo, Y. S. You, J. Wong, K. W. Wong, C. C. Wang, K. H. Lee, K. Yamane, V. B. Naik, C. S. Seet, T. Tahmasebi, C. Hai, H. W. Yang, N. Thiyagarajah, R. Chao, J. W. Ting, N. L. Chung, T. Ling, T. H. Chan, S. Y. Siah, R. Nair, S. Deshpande, R. Whig,...

    2017

  29. [38]

    The Memory Wall and Its Implications

    SiliconMatter 2024. The Memory Wall and Its Implications. https://siliconmatter.substack.com/p/the-memory-wall-and-its- implications

    2024

  30. [39]

    Devesh Singh and Donald Yeung. 2024. MORSE: Memory Overwrite Time Guided Soft Writes to Improve ReRAM Energy and Endurance. In Proceedings of the 2024 International Conference on Parallel Architec- tures and Compilation Techniques (Long Beach, CA, USA) (PACT ’24). Association ...

    2024

  31. [40]

    Loh, Michael J

    Alan Smith, Gabriel H. Loh, Michael J. Schulte, Mike Ignatowski, Samuel Naffziger, Mike Mantor, Mark Fowler Nathan Kalyanasund- haram, Vamsi Alla, Nicholas Malaya, Joseph L. Greathouse, Eric Chap- man, and Raja Swaminathan. 2024. Realizing the AMD Exascale Heterogeneous Proces...

    2024

  32. [41]

    Smullen, Vidyabhushan Mohan, Anurag Nigam, Sudhanva Gurumurthi, and Mircea R

    Clinton W. Smullen, Vidyabhushan Mohan, Anurag Nigam, Sudhanva Gurumurthi, and Mircea R. Stan. 2011. Relaxing non-volatility for fast and energy-efficient STT-RAM caches. In 2011 IEEE 17th International Symposium on High Performance Computer Architecture . 50–61. https: //doi....

    2011

  33. [42]

    Spheron 2024. How Much GPU Memory is Required to Run a Large Language Model? Find Out Here! https://blog.spheron.network/how- much-gpu-memory-is-required-to-run-a-large-language-model- find-out-here

    2024

  34. [43]

    DAM: Differentiated Access Memory Systems and Ap- plications

    Stanford 2024. DAM: Differentiated Access Memory Systems and Ap- plications. https://dam.stanford.edu/assets/Stanford_DAM_2_Pages_ 2024.pdfl

    2024

  35. [44]

    Jovan Stojkovic, Chaojie Zhang, Íñigo Goiri, Esha Choukse, Haoran Qiu, Rodrigo Fonseca, Josep Torrellas, and Ricardo Bianchini. 2025. TAPAS: Thermal- and Power-Aware Scheduling for LLM Inference in Cloud Platforms. arXiv:2501.02600 [cs.DC] https://arxiv.org/abs/2501. 02600

    2025 arXiv

  36. [45]

    Foteini Strati, Sara Mcallister, Amar Phanishayee, Jakub Tarnawski, and Ana Klimovic. 2024. D \’ej\aVu: KV-cache Streaming for Fast, Fault-tolerant Generative LLM Serving.arXiv preprint arXiv:2403.01876 (2024)

    2024 arXiv

  37. [46]

    2013.Exploring Memory Hierarchy Design with Emerging Memory Technologies

    Guangyu Sun. 2013.Exploring Memory Hierarchy Design with Emerging Memory Technologies. Springer Publishing Company, Incorporated

    2013

  38. [47]

    Zhenyu Sun, Xiuyuan Bi, Hai Li, Weng-Fai Wong, Zhong-Liang Ong, Xiaochun Zhu, and Wenqing Wu. 2011. Multi retention level STT-RAM cache designs with a dynamic refresh scheme. In 2011 44th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO) . 329–338

    2011

  39. [48]

    Yupeng Tang, Runxiang Cheng, Ping Zhou, Tongping Liu, Fei Liu, Wei Tang, Kyoungryun Bae, Jianjun Chen, Wu Xiang, and Rui Shi. [n. d.]. Exploring CXL-based KV Cache Storage for LLM Serving. ([n. d.])

  40. [49]

    Micron Plans HBM4E in 2028

    TomsHardware 2023. Micron Plans HBM4E in 2028. https://www.tomshardware.com/pc-components/ddr5/micron- plans-hbm4e-in-2028-256gb-ddr5-12800-ram-sticks-in-2026

    2023

  41. [50]

    Nvidia’s next-gen AI GPU is 4X faster than Hopper: Blackwell B200 GPU delivers up to 20 petaflops of compute and other massive improvements

    TomsHardware.com 2024. Nvidia’s next-gen AI GPU is 4X faster than Hopper: Blackwell B200 GPU delivers up to 20 petaflops of compute and other massive improvements. https://www.tomshardware.com/pc- components/gpus/nvidias-next-gen-ai-gpu-revealed-blackwell- b200-gpu-delivers-up...

    2024

  42. [51]

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Ł ukasz Kaiser, and Illia Polosukhin. 2017. Attention is All you Need. In Advances in Neural Information Pro- cessing Systems , I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S...

    2017

  43. [52]

    Naveen Verma, Hongyang Jia, Hossein Valavi, Yinqi Tang, Murat Ozatay, Lung-Yen Chen, Bonan Zhang, and Peter Deaville. 2019. In- memory computing: Advances and prospects. IEEE Solid-State Circuits Magazine 11, 3 (2019), 43–55

    2019

  44. [53]

    Automatic Prefix Caching

    vLLM.ai 2024. Automatic Prefix Caching. https://docs.vllm.ai/en/ latest/features/automatic_prefix_caching.html

    2024

  45. [54]

    Run-Jin Wu, Feng Chen, Cheng-Jer Yang, Feng Xu, OneGyun Na, and Ying-Qi Yang. 2022. A Fully Parallel On-Die ECC Architecture 7 Sergey Legtchenko, Ioan Stefanovici, Richard Black, Antony Rowstron, Junyi Liu, Paolo Costa, Burcu Canakci, Dushyanth Narayanan, Xingbo Wu with High A...

    2022

  46. [55]

    Hanmei Yang, Jin Zhou, Yao Fu, Xiaoqun Wang, Ramine Roane, Hui Guan, and Tongping Liu. 2024. ProTrain: Efficient LLM Training via Memory-Aware Techniques. arXiv preprint arXiv:2406.08334 (2024)

    2024 arXiv

  47. [56]

    Penghao Zhao, Hailin Zhang, Qinhan Yu, Zhengren Wang, Yunteng Geng, Fangcheng Fu, Ling Yang, Wentao Zhang, Jie Jiang, and Bin Cui

  48. [57]

    SSDs with NVMe Zoned Namespace (ZNS) Support

    zonedstorage.io 2019. SSDs with NVMe Zoned Namespace (ZNS) Support. https://zonedstorage.io/docs/introduction/zns. 8

    2019

  49. [546]

    https://doi.org/10.1109/MUE.2008.61

    2008 doi

  50. [2024]

    arXiv:2402.19473 [cs.CV] https://arxiv.org/abs/2402.19473

    Retrieval-Augmented Generation for AI-Generated Content: A Survey. arXiv:2402.19473 [cs.CV] https://arxiv.org/abs/2402.19473

    arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.