REVIEW 4 major objections 5 minor 31 references
AXLE: Coordinated Offloading with Asynchronous Back-Streaming in Computational Memory Systems
T0 review · 4 major / 5 minor · reviewed 2026-08-03 · deepseek-v4-flash
Pith's one-line read Letting a CXL memory device push results into host memory, instead of the host fetching them, overlaps computation with data movement and cuts end-to-end runtime by up to 50.4%.
desk verdict A genuinely new CXL offload protocol with a plausible 50% win—on the condition that a CXL Type 3 device can actually act as a DMA bus master, which the paper does not demonstrate. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is the asynchronous back-streaming protocol, a split of data and control over CXL: data flows device-to-host via CXL.io DMA posted writes initiated by the device, while control flows host-to-device via low-overhead CXL.mem stores. Two host-local ring buffers—one for payload, one for metadata—let the host poll a single local tail pointer; each metadata entry stores a payload slot identifier, enabling gap-aware, out-of-order consumption. The enabling assumption is a DMA engine attached as a bus master on a CXL Type 3 device, which lets the device initiate the reverse transfer without changing the CXL protocol.
What would settle it
Measure on a real CXL Type 3 device with a bus-master DMA engine whether device-initiated CXL.io posted writes to host physical addresses complete at the assumed latency and preserve ordering relative to a following metadata-tail update; if writes are slow, reordered, or unsupported, the claimed overlap and idle-time reductions will not reproduce.
Extended reading notes
Core claim
The authors first show that the two established offloading mechanisms are each lopsided. Remote polling over CXL.io allows asynchronous execution but carries polling intervals that dominate fine-grained tasks, while bulk synchronous flow over CXL.mem has low launch overhead but stalls the host until the remote kernel finishes and results are loaded. They then propose a third protocol. In asynchronous back-streaming, the computational memory device monitors its own result output and, once a configurable amount of data is ready, uses a DMA engine attached as a bus master on a CXL Type 3 device to issue CXL.io posted writes into host-local ring buffers. The host polls only a single local metada
Load-bearing premise
The whole result rests on a CXL memory device being able to initiate fast DMA writes into host memory by itself; if device-initiated CXL.io posted writes at the assumed 350 ns latency do not work on real hardware, the back-streaming mechanism cannot happen and the runtime gains disappear.
Editorial extensions
If this is right
- Offloading fine-grained kernels (microsecond-scale attention blocks, distance computations, filters) becomes viable because notification cost is a local pointer poll rather than a CXL round-trip.
- Host cores no longer stall on result loads; reported host idle time drops 3.85x on average, freeing those cores for other tasks or allowing fewer cores per server.
- Data-movement-heavy workloads such as graph analytics see the largest gains, up to 50.4% end-to-end, because streaming hides the transfer behind compute.
- The benefit depends on small streaming factors; batching results into large DMA transfers erodes overlap and can make the protocol slower than bulk synchronous flow.
- Out-of-order streaming is required for the advertised gains: disabling it increases runtime by up to 2.28x under round-robin scheduling.
Reading between the lines
- If the assumed 350 ns device-initiated CXL.io write latency holds on real hardware, the protocol needs no new CXL feature, only a DMA engine with bus-master capability; the main unknown on the path to silicon is whether memory-fence ordering between payload writes and metadata-tail updates is preserved.
- The authors leave implicit that back-streaming makes the host less sensitive to CXL link latency in general: only flow-control messages sit on the critical path, so slower or more congested links degrade overlap less than in fetch-based models.
- A natural extension is an adaptive streaming factor and polling interval chosen per workload phase; the paper's experiments show both parameters have a strong effect on end-to-end runtime, so a self-tuning version of the proposed system should outperform fixed settings.
- For LLM inference, the paper finds marginal gains when the host has few tasks and the final task tail dominates; a testable fix is splitting the last host task or batching multiple inference requests, which would expose the protocol's overlap in pipeline-shaped workloads.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper studies partial offloading to CXL-based Computational Memory (CCM) devices. It contrasts two existing mechanisms: remote polling over CXL.io, which is asynchronous but high-overhead, and bulk synchronous flow over CXL.mem, which is low-overhead but blocks the host. It proposes asynchronous back-streaming, in which the CCM device uses a DMA engine to stream partial results into a host-local ring buffer, with local polling and flow control via CXL.mem writes. The KAI system implements this protocol on top of the M2NDP simulator. Evaluations on eight workloads claim up to 50.4% end-to-end runtime reduction and large idle-time reductions relative to the two baselines.
Significance. If the key hardware assumption—that a CXL Type 3 device can initiate CXL.io posted writes to host physical addresses through an attached DMA bus master—is valid, the paper identifies a genuine inefficiency in CCM offloading pipelines and offers a plausible, well-structured solution. The trade-off analysis in §III is useful, and the separation of metadata and payload ring buffers plus out-of-order streaming is a thoughtful systems design. The use of an open-source simulator and the detailed implementation description are strengths. However, the central quantitative claims rest on an unverified hardware capability and are validated only in a simulator; the paper is best read as a design study pending hardware confirmation.
major comments (4)
- [IV-D] Section IV-D states: 'To enable device-initiated data transfers ... we target environments where a DMA engine is attached as a bus master on top of a CXL Type 3 device. In this configuration, payloads are transferred from the device to the host physical address via a CXL.io (PCIe) posted write.' This is the linchpin of the entire asynchronous back-streaming protocol: without this device-initiated DMA capability, the back-streaming shown in Fig. 1(c) and all subsequent gains (up to 50.4% in Fig. 10(e)) cannot occur. The paper gives no evidence that this configuration is permitted by the CXL specification, exists in any current CXL controller, or has been demonstrated in the authors' own FPGA prototype. The CXL.io latency is only configured as 350 ns in Table III. The authors should provide a concrete specification reference or prototype evidence, or explicitly label the results as conditi
- [IV-C] The memory-correctness section asserts that 'our simulator implementation enforces this ordering and verifies functional correctness while running applications.' The critical ordering between payload writes and metadata tail updates is enforced only in the simulator. On real hardware, posted writes over PCIe/CXL.io do not guarantee this ordering without an explicit fence or completion mechanism. The paper should identify the specific hardware mechanism—e.g., a DMA write barrier, a read-after-write to the same location, or a CXL protocol rule—that enforces 'payload data write → metadata tail index update' on actual CXL devices. Without this, the consistency invariant is unsubstantiated and the ring-buffer protocol may expose partial or stale data to the host.
- [V-A] The evaluation is based on a single simulation run for each configuration. The paper does not state whether the M2NDP simulator is deterministic, nor does it report error bars or confidence intervals. The DMA preparation time (500 ns) and CXL.io round-trip latency (350 ns) in Table III are assumed values with no sensitivity analysis. Since the paper's headline claims are precise quantitative reductions, the authors should either confirm that the simulator is deterministic and report the exact configuration, or, if stochastic, provide multiple runs with error bars. Sensitivity to the assumed DMA overhead should also be discussed, as it directly affects the claimed speedups.
- [II] The paper itself states that 'this simulation-based research serves as a preparatory step toward realizing and validating the new data and control planes on an upcoming ASIC-based CCM device.' Thus, the abstract and conclusion's unqualified statements that KAI 'reduces end-to-end runtime by up to 50.4%' overstate what has been demonstrated. The quantitative gains should be presented as simulation estimates conditioned on the DMA hardware assumption, not as measured system performance. Weighing the manuscript's own admission, this is a load-bearing limitation for the central claim.
minor comments (5)
- [Abstract vs. body] The system is called AXLE in the abstract and arXiv title, but KAI throughout the body. The abstract also reports '50.14%' while §VII reports '50.4%'. These inconsistencies should be reconciled.
- [Table III] The 'DMA preparation time per request: 500 ns' parameter is given without a source or measurement. Please add a justification or reference.
- [Figure 10(h)] The LLM case is an important negative result where KAI provides little benefit. Consider discussing this case more prominently in the conclusion to temper the general claims.
- [IV-D] The related-work discussion could cite existing CXL ordering rules or DMA implementations on CXL devices to strengthen the motivation in §IV-D.
- [IV-C] The description of the 'visibility problem' would benefit from a short example or pseudo-code for the ring-buffer head/tail arithmetic, as the current text is dense.
Circularity Check
No significant circularity: the 50.4% claim is a simulator output produced from explicitly configured latencies, not a fitted or self-referential quantity.
full rationale
The paper's central result — KAI reducing end-to-end runtime by up to 50.4% and idle times by 22.11x/3.85x — is obtained from cycle-level simulation on the M2NDP simulator (Section V). No equation in the paper fits a parameter to the target result and then re-presents it as a prediction. The DMA latency (350 ns), polling intervals, and DMA preparation time (500 ns) are configured assumptions in Table III, not constants fitted to reproduce the claimed reductions. The key hardware precondition — a DMA engine attached as a bus master to a CXL Type 3 device — is stated as a targeted environment in Section IV-D ('we target environments where a DMA engine is attached as a bus master on top of a CXL Type 3 device'), not derived from the results. This is an unverified feasibility assumption and a correctness risk, but it is not circular: if the hardware cannot perform device-initiated CXL.io posted writes, the results fail, but the simulation still derives its outputs from that explicitly stated premise rather than from the outputs themselves. The paper also openly labels the work a 'preparatory step toward realizing and validating the new data and control planes on an upcoming ASIC-based CCM device' (Section II), which further clarifies that hardware validation is pending, not hidden. The self-citations to [19] and [31] (sharing authors with this paper) are used only to motivate graph-analytics offloading and frontier-dependency observations (Sections III-B, III-C); they are not used to justify the asynchronous back-streaming protocol or to supply its predicted performance, so they are not load-bearing. The M2NDP baseline itself is an external open-source simulator [9] cited as independent support. Memory-fence ordering is asserted and verified only in the simulator (Section IV-C), which is a validation limitation, but no circular step reduces a prediction to its own inputs. Overall, no derivation chain in the paper equates a fitted input with a claimed prediction or imports a self-authored uniqueness theorem to force the design. The appropriate finding is no significant circularity.
Assumptions & free parameters
free parameters (5)
- DMA preparation time per request =
500 ns
- Local polling interval (default p1) =
50 ns
- Streaming factor (SF1 default) =
32 B
- CXL.io / CXL.mem round-trip latencies =
350 ns / 70 ns
- Interrupt handling time (KAI_Interrupt baseline) =
50 us
assumptions (5)
- domain assumption A CXL Type 3 device can host a DMA engine that writes host physical memory via CXL.io posted writes
- domain assumption M2NDP simulator is a valid CCM model and can faithfully represent the new DMA/control plane
- domain assumption Memory fences in the simulator enforce the required ordering invariants
- domain assumption Cache-bypass pinned DMA regions do not hurt performance
- domain assumption The eight evaluated workloads and their offloading targets represent the design space
invented entities (2)
-
KAI DMA executor
-
Metadata/payload ring-buffer pair in host DMA region
Cite this review
Pith. "Pith review of AXLE: Coordinated Offloading with Asynchronous Back-Streaming in Computational Memory Systems." pith.science (2026). https://pith.science/paper/GOABVNTP
@misc{pith2026251204449,
author = {Pith},
title = {Pith review of: AXLE: Coordinated Offloading with Asynchronous Back-Streaming in Computational Memory Systems},
year = {2026},
howpublished = {\url{https://pith.science/paper/GOABVNTP}},
note = {Machine review of arXiv:2512.04449}
}
read the original abstract
CXL-based Computational Memory (CCM) enables near-memory processing within expanded remote memory, offering opportunities to address data movement costs in disaggregated memory systems and to accelerate overall performance. However, existing offloading mechanisms do not fully leverage the trade-offs of different offload models based on different CXL protocols. This work first examines these tradeoffs and their impact on end-to-end performance and system efficiency for workloads with diverse data and computation characteristics. We propose Asynchronous Back-Streaming, a new offloading protocol that coordinates CXL.io and CXL.mem to enable result back-streaming and asynchronous pipelining across CCM and host tasks. We further design AXLE, a system that realizes this protocol with lightweight host-CCM interaction. Overall, AXLE reduces end-to-end runtime by up to 50.14%, reduces CCM and host idle times by an average of 14.53x and 3.93x, respectively, and achieves up to 6x reduction in host core stall time.
Figures
Figures from the paper (9 more)
Reference graph
Works this paper leans on
-
[1]
Memory disaggregation: why now and what are the challenges,
M. K. Aguilera, E. Amaro, N. Amit, E. Hunhoff, A. Yelam, and G. Zellweger, “Memory disaggregation: why now and what are the challenges,”SIGOPS Oper. Syst. Rev., vol. 57, no. 1, p. 38–46, jun 2023
2023
-
[2]
Remote memory in the age of fast networks,
M. K. Aguilera, N. Amit, I. Calciu, X. Deguillard, J. Gandhi, P. Sub- rahmanyam, L. Suresh, K. Tati, R. Venkatasubramanian, and M. Wei, “Remote memory in the age of fast networks,” inProceedings of the 2017 Symposium on Cloud Computing, ser. SoCC ’17. New York, NY , USA: Association for Computing Machinery, 2017, p. 121–127
2017
-
[3]
Cxl 3.0 specification,
Compute-Express-Link-Consortium, “Cxl 3.0 specification,” https:// www.computeexpresslink.org/download-the-specification, 2022, (On- line; downloaded 2023)
2022
-
[4]
An introduction to the compute express link (cxl) interconnect,
D. Das Sharma, R. Blankenship, and D. Berger, “An introduction to the compute express link (cxl) interconnect,”ACM Comput. Surv., vol. 56, no. 11, Jul. 2024
2024
-
[5]
Direct access, High- Performance memory disaggregation with DirectCXL,
D. Gouk, S. Lee, M. Kwon, and M. Jung, “Direct access, High- Performance memory disaggregation with DirectCXL,” in2022 USENIX Annual Technical Conference (USENIX ATC 22). Carlsbad, CA: USENIX Association, Jul. 2022, pp. 287–294
2022
-
[6]
Efficient memory disaggregation with infiniswap,
J. Gu, Y . Lee, Y . Zhang, M. Chowdhury, and K. G. Shin, “Efficient memory disaggregation with infiniswap,” in14th USENIX Symposium on Networked Systems Design and Implementation (NSDI 17). Boston, MA: USENIX Association, Mar. 2017, pp. 649–667
2017
-
[7]
The benefits and limitations of user interrupts for preemptive userspace scheduling,
L. Guo, D. Zuberi, T. Garfinkel, and A. Ousterhout, “The benefits and limitations of user interrupts for preemptive userspace scheduling,” in Proceedings of the 22nd USENIX Symposium on Networked Systems De- sign and Implementation, ser. NSDI ’25. USA: USENIX Association, 2025
2025
-
[8]
Low-overhead general-purpose near- data processing in cxl memory expanders,
H. Ham, J. Hong, G. Park, Y . Shin, O. Woo, W. Yang, J. Bae, E. Park, H. Sung, E. Lim, and G. Kim, “Low-overhead general-purpose near- data processing in cxl memory expanders,” in2024 57th IEEE/ACM International Symposium on Microarchitecture (MICRO), 2024, pp. 594– 611
2024
Show all 31 references
-
[9]
M²ndp: A cycle-level simulator for memory-mapped near-data processing,
——, “M²ndp: A cycle-level simulator for memory-mapped near-data processing,” GitHub repository, Pohang, South Korea, 2024, accessed: 2025-08-20. [Online]. Available: https://github.com/PSAL-POSTECH/ M2NDP-public
2024
-
[10]
Neupims: Npu-pim heterogeneous acceleration for batched llm inferencing,
G. Heo, S. Lee, J. Cho, H. Choi, S. Lee, H. Ham, G. Kim, D. Ma- hajan, and J. Park, “Neupims: Npu-pim heterogeneous acceleration for batched llm inferencing,” inProceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operatin...
2024
-
[11]
Udon: A case for offloading to general purpose compute on cxl memory,
J. Hermes, J. Minor, M. Wu, A. Patil, and E. V . Hensbergen, “Udon: A case for offloading to general purpose compute on cxl memory,” 2024
2024
-
[12]
Beacon: Scalable near-data-processing accelerators for genome analysis near memory pool with the cxl support,
W. Huangfu, K. T. Malladi, A. Chang, and Y . Xie, “Beacon: Scalable near-data-processing accelerators for genome analysis near memory pool with the cxl support,” in2022 55th IEEE/ACM International Symposium on Microarchitecture (MICRO), 2022, pp. 727–743
2022
-
[13]
Pifs-rec: Process- in-fabric-switch for large-scale recommendation system inferences,
P. Huo, A. Devulapally, H. A. Maruf, M. Park, K. Nair, M. Arunachalam, G. G. Akbulut, M. T. Kandemir, and V . Narayanan, “Pifs-rec: Process- in-fabric-switch for large-scale recommendation system inferences,” in 2024 57th IEEE/ACM International Symposium on Microarchitecture (...
2024
-
[14]
CXL- ANNS: Software-Hardware collaborative memory disaggregation and computation for Billion-Scale approximate nearest neighbor search,
J. Jang, H. Choi, H. Bae, S. Lee, M. Kwon, and M. Jung, “CXL- ANNS: Software-Hardware collaborative memory disaggregation and computation for Billion-Scale approximate nearest neighbor search,” in 2023 USENIX Annual Technical Conference (USENIX ATC 23). Boston, MA: USENIX Asso...
2023
-
[15]
A detailed and flexible cycle-accurate network-on-chip simulator,
N. Jiang, D. U. Becker, G. Michelogiannakis, J. Balfour, B. Towles, D. E. Shaw, J. Kim, and W. J. Dally, “A detailed and flexible cycle-accurate network-on-chip simulator,” in2013 IEEE International Symposium on Performance Analysis of Systems and Software (ISPASS), 2013, pp. 86– 96
2013
-
[16]
Monde: Mixture of near-data experts for large-scale sparse models,
T. Kim, K. Choi, Y . Cho, J. Cho, H.-J. Lee, and J. Sim, “Monde: Mixture of near-data experts for large-scale sparse models,” inProceedings of the 61st ACM/IEEE Design Automation Conference, ser. DAC ’24. New York, NY , USA: Association for Computing Machinery, 2024
2024
-
[17]
Ramulator: A fast and extensible dram simulator,
Y . Kim, W. Yang, and O. Mutlu, “Ramulator: A fast and extensible dram simulator,”IEEE Comput. Archit. Lett., vol. 15, no. 1, p. 45–49, Jan. 2016. 12
2016
-
[18]
COSMOS: A CXL-Based Full In-Memory System for Approximate Nearest Neighbor Search,
S. Ko, H. Shim, W. Doh, S. Yun, J. So, Y . Kwon, S.-S. Park, S.-D. Roh, M. Yoon, T. Song, and J. H. Ahn, “COSMOS: A CXL-Based Full In-Memory System for Approximate Nearest Neighbor Search,”IEEE Computer Architecture Letters, no. 01, pp. 1–4, May 5555
-
[19]
Towards disaggregated ndp architectures for large-scale graph analytics,
S. Lee, V . Rao, and A. Gavrilovska, “Towards disaggregated ndp architectures for large-scale graph analytics,” inProceedings of the SC ’24 Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis, ser. SC-W ’24. IEEE Press, 2025,...
2025
-
[20]
Pond: Cxl-based memory pooling systems for cloud platforms,
H. Li, D. S. Berger, L. Hsu, D. Ernst, P. Zardoshti, S. Novakovic, M. Shah, S. Rajadnya, S. Lee, I. Agarwal, M. D. Hill, M. Fontoura, and R. Bianchini, “Pond: Cxl-based memory pooling systems for cloud platforms,” inProceedings of the 28th ACM International Conference on Archi...
2023
-
[21]
A fine-grain multithreading super- scalar architecture,
M. Loikkanen and N. Bagherzadeh, “A fine-grain multithreading super- scalar architecture,” inProceedings of the 1996 Conference on Parallel Architectures and Compilation Techniques, ser. PACT ’96. USA: IEEE Computer Society, 1996, p. 163
1996
-
[22]
Tpp: Transparent page placement for cxl-enabled tiered-memory,
H. A. Maruf, H. Wang, A. Dhanotia, J. Weiner, N. Agarwal, P. Bhat- tacharya, C. Petersen, M. Chowdhury, S. Kanaujia, and P. Chauhan, “Tpp: Transparent page placement for cxl-enabled tiered-memory,” in Proceedings of the 28th ACM International Conference on Architectural Suppor...
2023
-
[23]
O’Neil, E
P. O’Neil, E. O’Neil, X. Chen, and S. Revilak,The Star Schema Benchmark and Augmented Fact Table Indexing. Berlin, Heidelberg: Springer-Verlag, 2009, p. 237–252
2009
-
[24]
Shenango: Achieving high CPU efficiency for latency-sensitive data- center workloads,
A. Ousterhout, J. Fried, J. Behrens, A. Belay, and H. Balakrishnan, “Shenango: Achieving high CPU efficiency for latency-sensitive data- center workloads,” in16th USENIX Symposium on Networked Systems Design and Implementation (NSDI 19). Boston, MA: USENIX Asso- ciation, Feb. ...
2019
-
[25]
An lpddr-based cxl-pnm platform for tco-efficient inference of transformer-based large language models,
S.-S. Park, K. Kim, J. So, J. Jung, J. Lee, K. Woo, N. Kim, Y . Lee, H. Kim, Y . Kwon, J. Kim, J. Lee, Y . Cho, Y . Tai, J. Cho, H. Song, J. H. Ahn, and N. S. Kim, “An lpddr-based cxl-pnm platform for tco-efficient inference of transformer-based large language models,” in2024 ...
2024
-
[26]
Nearpm: A near- data processing system for storage-class applications,
Y . Seneviratne, K. Seemakhupt, S. Liu, and S. Khan, “Nearpm: A near- data processing system for storage-class applications,” inProceedings of the Eighteenth European Conference on Computer Systems, ser. Eu- roSys’23. New York, NY , USA: Association for Computing Machinery, 20...
2023
-
[27]
Strom: Smart remote memory,
D. Sidler, Z. Wang, M. Chiosa, A. Kulkarni, and G. Alonso, “Strom: Smart remote memory,” inProceedings of the Fifteenth European Conference on Computer Systems, ser. EuroSys’20. New York, NY , USA: Association for Computing Machinery, 2020
2020
-
[28]
Computational cxl-memory solution for accel- erating memory-intensive applications,
J. Sim, S. Ahn, T. Ahn, S. Lee, M. Rhee, J. Kim, K. Shin, D. Moon, E. Kim, and K. Park, “Computational cxl-memory solution for accel- erating memory-intensive applications,”IEEE Computer Architecture Letters, vol. 22, no. 1, pp. 5–8, 2022
2022
-
[29]
Demystifying cxl memory with genuine cxl-ready systems and devices,
Y . Sun, Y . Yuan, Z. Yu, R. Kuper, I. Jeong, R. Wang, and N. S. Kim, “Demystifying cxl memory with genuine cxl-ready systems and devices,” 2023
2023
-
[30]
Clay: Cxl-based scalable ndp architecture accelerating embedding layers,
S. Yun, H. Nam, K. Kyung, J. Park, B. Kim, Y . Kwon, E. Lee, and J. H. Ahn, “Clay: Cxl-based scalable ndp architecture accelerating embedding layers,” inProceedings of the 38th ACM International Conference on Supercomputing, ser. ICS ’24. New York, NY , USA: Association for Co...
2024
-
[31]
Fam-graph: Graph analytics on disag- gregated memory,
D. Zahka and A. Gavrilovska, “Fam-graph: Graph analytics on disag- gregated memory,” in2022 IEEE International Parallel and Distributed Processing Symposium (IPDPS), 2022, pp. 81–92. 13
2022
Reviewed August 3, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.