REVIEW 3 major objections 6 minor 1 cited by
Memory Access Vectors: Improving Sampling Fidelity for CPU Performance Simulations
T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read By adding Memory Access Vectors to SimPoint's Basic Block Vectors, this paper lifts the projected-score correlation for 523.xalancbmk_r from 0.80 to 0.98 on a 192-core SoC.
desk verdict A plausible and genuinely new sampling vector, but the single benchmark result doesn't yet prove the method — worth referee time. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Memory Access Vector (MAV) is the central object: for each instruction window, count read and write accesses to unique 4096-byte physical-memory regions under functional execution, with no cache or TLB dependence. The processing pipeline then ranks regions by inverse access frequency so that rarely touched, miss-prone regions dominate, normalizes the whole MAV matrix by average row magnitude to preserve memory-intensity differences between phases, applies a 0.95 exponential decay over the previous ten windows for temporal locality, projects both BBV and MAV matrices to 15 dimensions with Gaussian random projection, and concatenates them into a 30-dimensional signature. The adaptive weighting scales the MAV contribution by the application-wide fraction of memory operations, so code-heavy applications stay BBV-dominated while data-heavy applications give memory behavior a strong vote. The combined matrix goes into SimPoint's k-means clustering to pick representative windows.
What would settle it
Run a complete (or much longer) simulation of 523.xalancbmk_r on the performance model and compare its SPECrate projection to AmpereOne silicon; if the full-trace model still lands near 0.80 correlation, the claimed improvement is an artifact of sample reweighting rather than a fidelity fix. A cheaper check is to apply BBV+MAV to another array-indirect benchmark and see whether the correlation gain over BBV-only sampling persists.
Extended reading notes
Core claim
The paper's central claim is that SimPoint's sampling error on 523.xalancbmk_r is a signature problem, not a model problem: BBVs encode which code runs but not which memory is touched, so recurring code with shifting data phases looks homogeneous. To test this, the authors collect MAV histograms for the benchmark's 980 billion instructions and build a 30-cluster SimPoint with BBV+MAV. BBV alone assigned two phases to the first 200 billion instructions (the Xerces parser region), while BBV+MAV assigns twelve, and the new phase boundaries line up with IPC variation measured on silicon. Re-running the standard projection flow with these new SimPoints raises the SPECrate correlation for 523.xalancbmk_r from 0.80 to 0.98 on the 192-core SoC and from 0.84 to 0.95 at 96 cores. The paper presents this as evidence that memory-access-aware sample selection, not a change to the performance model, closes the projection gap.
Load-bearing premise
The load-bearing premise is that the 20% projection gap for 523.xalancbmk_r comes from sample selection rather than from the performance model's own simulation error, since no full-trace run independently validates the model on that benchmark.
Editorial extensions
If this is right
- For 523.xalancbmk_r, projected SPECrate correlation with AmpereOne silicon rises from 0.80 to 0.98 on the 192-core SoC and from 0.84 to 0.95 on the 96-core part.
- The 30-cluster SimPoint configuration remains valid for BBV+MAV, so existing heuristics for choosing cluster counts carry over unchanged.
- MAV collection adds only histogram-update work during the emulation-based preprocessing step; trace collection after SimPoint selection is unaffected.
- The combined vectors split the Xerces parser region into 12 phases rather than 2, matching silicon IPC variation, which is why the old samples underrepresented it.
- The authors expect the same approach to help other workloads with extensive array-indirect accesses, such as graph analytics and machine-learning inference.
Reading between the lines
- A testable extension would apply BBV+MAV to 623.xalanc_s or to other SPEC CPU2017 rates with large data footprints; if the correlation gain does not reproduce there, the effect may be specific to 523.xalanc's parser/transformer split.
- Because MAV uses absolute physical addresses at 4096-byte granularity, phase signatures may depend on memory layout and page allocation; hashing or virtualizing addresses before building histograms would test whether the improvement is layout-independent.
- The matrix-level normalization and temporal-decay terms suggest the method could carry over to projections of multi-program or many-core SoCs, where memory pressure from co-running workloads is part of the phase signal.
- If the method generalizes, published sampling studies that exclude 523.xalanc due to convergence problems could be repeated with MAV, letting the benchmark return to standard projection suites.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Memory Access Vectors (MAV), a microarchitecture-independent representation of per-window memory access histograms, and integrates it with SimPoint's Basic Block Vectors (BBV) for phase detection. The combined BBV+MAV signature uses inverse-frequency sorting, matrix-level normalization, temporal decay, Gaussian random projection to 15 dimensions, and adaptive weighting by memory-op fraction. The authors apply this to 523.xalancbmk_r on an AmpereOne SoC and report that the projected-to-silicon ratio (which they call "correlation") improves from 0.80 to 0.98 at 192 cores and from 0.84 to 0.95 at 96 cores (Table II). They support this with recurrence plots, phase plots, and a silicon IPC overlay (Figures 1-4).
Significance. If the improvement is real and robust, the paper addresses a known gap: SimPoint-based sampling can fail on workloads with array-indirect memory accesses, and 523.xalancbmk_r has previously been excluded from evaluation due to convergence problems. The MAV idea is simple, microarchitecture-independent, and the implementation via QEMU is lightweight. The paper also explicitly motivates the problem for large server-class CPUs with many cores. However, the evidence is currently limited to a single benchmark and a single SoC configuration, with no sensitivity analysis for the many introduced hyperparameters and no direct sampling-only validation; the reported metric conflates sampling fidelity with performance-model bias. The contribution is potentially useful for the performance-evaluation community, but it is not yet established at the level claimed.
major comments (3)
- [Section IV-D, Table II] The central quantitative claim compares the performance model's projected SPECrate ratio against silicon, not sampling fidelity. The baseline 0.80 comes from the same model (Table I); if the model systematically mispredicts the Xerces parser region, shifting more samples into that region, as BBV+MAV does, could increase the projected score for reasons unrelated to how well the samples represent the full program. The paper lacks a sampling-only validation, such as comparing the weighted IPC or cache-miss rates of the selected samples against full-trace silicon measurements. The qualitative overlay in Figure 4 is suggestive but not quantified. Without such validation, the 80% to 98% improvement cannot be attributed specifically to improved sampling fidelity.
- [Section III, processing flow items 1-5] The MAV pipeline introduces several tunable hyperparameters with no sensitivity analysis: the 4096-byte bucket granularity, the inverse-frequency sorting transformation, the 0.95 decay factor over 10 windows, the 15-dimensional random projection, and the memory-op weighting fraction. The single reported improvement could reflect parameter selection tailored to 523.xalancbmk_r rather than a robust property of the method. The authors should report results across a range of these parameters (at minimum varying the cluster count and random seed for k-means) to demonstrate that the improvement is not an artifact of one configuration.
- [Table I and Table II, Section IV-D] The values labeled "correlation" in Tables I and II are actually per-benchmark ratios of projected score to silicon (e.g., 0.84, 0.80, 0.98), not correlation coefficients. This terminology is misleading: the abstract's "increasing it from 80% to 98%" describes a reduction of projection error from 20% to 2%, not a statistical correlation. The authors should either rename these column headers to "projection ratio" or "accuracy" or, if they intend true correlations across benchmarks, provide the appropriate computation and clarify the difference.
minor comments (6)
- [Section III, item 1] The description of inverse-frequency sorting is ambiguous for memory regions with zero accesses; the inverse of zero is undefined. The authors should specify how non-accessed regions are handled (e.g., excluded, or assigned a maximum value).
- [Section IV-A] The paper says MAV records accesses in the "physical address space" but later states it is "independent of microarchitectural caches or TLBs." Since QEMU typically operates on virtual addresses, the authors should clarify whether they use virtual or physical addresses and how this choice affects the representation.
- [Section IV-B, Figure 1] The recurrence plots lack axis labels and a color scale, making it difficult to interpret the distance measure used. Please specify what distance metric (Euclidean, Manhattan, etc.) is plotted and the range of values represented.
- [Abstract and throughout] The text frequently renders "MAV" with a space as "MA V"; please use a consistent notation throughout.
- [Section IV-B] The description of Figure 1 says "the left image" and "the center image" but does not clearly refer to the three panels by number; labeling the panels (a), (b), (c) would improve readability.
- [References] Several references contain unintended line breaks or spacing artifacts (e.g., "V olume", "doi.org/ 10.1145"). Please format the bibliography cleanly.
Circularity Check
No circularity: the claimed 0.80→0.98 improvement is an empirical comparison of two sampling methods against external silicon measurements, not a quantity forced by construction or by self-citation.
full rationale
The central result in Table II compares the projected SPECrate correlation for 523.xalancbmk_r on AmpereOne silicon under BBV-only and BBV+MAV sampling. This is an empirical measurement against an external target (silicon), and no equation defines MAV in terms of the target correlation or fits a parameter to the reported 98%. The method in Section III specifies a fixed processing flow (inverse-frequency sorting, normalization, temporal decay, random projection, adaptive weighting, k-means clustering) before presenting the result, and the improvement is not statistically forced by construction: MAV could plausibly have selected worse samples. The paper's references to AmpereOne product materials are silicon data sources rather than load-bearing self-citations, and the in-house simulator is validated against external benchmarks. The assumption that the performance model is accurate for 523 except for sampling is a modeling assumption, not a circular reduction. Thus no specific circular step can be exhibited, and the appropriate finding is no significant circularity.
Assumptions & free parameters
free parameters (8)
- instruction window length =
10 million instructions
- memory bucket granularity =
4096 bytes
- temporal decay factor =
0.95
- temporal history length =
10 instruction windows
- random projection dimension for MAV =
15
- combined vector dimension =
30
- SimPoint cluster count =
30
- memory-op weighting fraction =
not stated
assumptions (6)
- standard math Gaussian Random Projection preserves the cluster structure at 15 dimensions.
- domain assumption Code-block frequency correlates with performance.
- ad hoc to paper The performance model is unbiased for 523.xalanc.
- ad hoc to paper Inverse-frequency sorting emphasizes performance-critical accesses.
- domain assumption SimPoint cluster-count heuristics remain valid after adding MAV.
- domain assumption QEMU functional memory addresses represent hardware-relevant memory behavior.
Cite this review
Pith. "Pith review of Memory Access Vectors: Improving Sampling Fidelity for CPU Performance Simulations." pith.science (2026). https://pith.science/paper/2E7SAM3E
@misc{pith2026250602344,
author = {Pith},
title = {Pith review of: Memory Access Vectors: Improving Sampling Fidelity for CPU Performance Simulations},
year = {2026},
howpublished = {\url{https://pith.science/paper/2E7SAM3E}},
note = {Machine review of arXiv:2506.02344}
}
read the original abstract
Accurate performance projection of large-scale benchmarks is essential for CPU architects to evaluate and optimize future processor designs. SimPoint sampling, which uses Basic Block Vectors (BBVs), is a widely adopted technique to reduce simulation time by selecting representative program phases. However, BBVs often fail to capture the behavior of applications with extensive array-indirect memory accesses, leading to inaccurate projections. In particular, the 523.xalancbmk_r benchmark exhibits complex data movement patterns that challenge traditional SimPoint methods. To address this, we propose enhancing SimPoint's BBV methodology by incorporating Memory Access Vectors (MAV), a microarchitecture independent technique that tracks functional memory access patterns. This combined approach significantly improves the projection accuracy of 523.xalancbmk_r on a 192-core system-on-chip, increasing it from 80% to 98%.
Figures
Forward citations
Cited by 1 Pith paper
-
CHIA: An open-source framework for principled, agentic AI-driven hardware/software co-design research
CHIA is an open-source framework for agentic AI-driven hardware/software co-design using CHIA loops as directed cyclic graphs, a tool library, and features for reliable experimentation, shown via five case studies.
Reference graph
Works this paper leans on
-
[1]
[Online] https://amperecomputing.com/briefs/ampereone-family-product-brief
AmpereOne® Product Brief (A192-32X), Ampere Computing. [Online] https://amperecomputing.com/briefs/ampereone-family-product-brief
-
[2]
D. Carlson, N. Simakov, R. Hadlich, A. Curtis, J. Martin, G. Verma, S. Chheda, F. Coskun, R. Gonzalez, D. Wood, F. Zhang, R. Harrison, E. Siegmann,The AmpereOne A192-32X in Perspective: Benchmarking a New Standard. In Proceedings of the 2025 International Conference on High Performance Computing in Asia-Pacific Region Workshops. Association for Computing ...
-
[3]
J. Bucek, K-D. Lange, J. Kistowski,SPEC CPU2017: Next-Generation Compute Benchmark, In Companion of the 2018 ACM/SPEC Interna- tional Conference on Performance Engineering (ICPE ’18). Association for Computing Machinery, New York, NY , USA. Available: https://doi. org/10.1145/3185768.3185771
arXiv 2018
-
[4]
T. Sherwood, E. Perelman, G. Hamerly, B. Calder,Automatically charac- terizing large scale program behavior, ACM International Conference on on Architectural Support for Programming Languages and Operating Sys- tems, October 2002. Available: https://api.semanticscholar.org/CorpusID: 6445892
work page 2002
-
[5]
G. Hamerly, E. Perelman, J. Lau, B. Calder,SimPoint 3.0: Faster and More Flexible Program Phase Analysis, Journal of Instruction- Level Parallelism, V olume 7, September 2005. Available: https://api. semanticscholar.org/CorpusID:11937761
work page 2005
-
[6]
B. Calder, T. Sherwood, G. Hamerly, E. Perelman,SimPoint: Pick- ing Representative Samples to Guide Simulation, book chapter from Processor and System-on-Chip Simulationedited by R. Leupers and O. Temam, Springer ISBN 978-1-4419-6174-7, 2010. Available: https: //link.springer.com/book/10.1007/978-1-4419-6175-4
-
[7]
S. Singh and M. Awasthi,Efficacy of Statistical Sampling on Con- temporary Workloads: The Case of SPEC CPU2017, IEEE Interna- tional Symposium on Workload Characterization (IISWC), Orlando, FL, USA, 2019, doi: 10.1109/IISWC47752.2019.9042114. Available: https: //ieeexplore.ieee.org/document/9042114
arXiv 2019
- [8]
Show all 27 references
-
[9]
J. Lau, J. Sampson, E. Perelman, G. Hamerly, B. Calder,The strong correlation between code signatures and performance, International Sym- posium on Performance Analysis of Systems and Software, March 2005. Available: https://ieeexplore.ieee.org/document/1430578
2005
-
[10]
Singhal, et.al,Performance Analysis and Validation of the Intel ® Pentium ® 4 Processor on 90 nm Technology, Intel Technology Journal,
R. Singhal, et.al,Performance Analysis and Validation of the Intel ® Pentium ® 4 Processor on 90 nm Technology, Intel Technology Journal,
- [11]
-
[12]
J. Jang, H. Kim and H. Lee,Characterizing Memory Access Patterns of Various Convolutional Neural Networks for Utilizing Processing-in- Memory, 2023 International Conference on Electronics, Information, and Communication (ICEIC), Singapore, 2023. Available: https://doi.org/10. ...
2023
-
[13]
Balaji, N
V . Balaji, N. Crago, A. Jaleel and B. Lucia,P-OPT: Practical Optimal Cache Replacement for Graph Analytics, 2021 IEEE International Sym- posium on High-Performance Computer Architecture (HPCA), Seoul, Ko- rea (South), 2021. Available: https://doi.org/10.1109/HPCA51647.2021. 00062
2021
-
[14]
V . Weaver,exp-bbv: Valgrind plugin that makes SimPoint Basic Block Vector Files, [Online] https://valgrind.org/docs/manual/bbv-manual.html and https://web.eece.maine.edu/~vweaver/projects/valsim
-
[15]
Weaver,qemu_bbv - a qemu patch that enables SimPoint Basic Block Vector File Generation, [Online] https://web.eece.maine.edu/~vweaver/ projects/qemusim
V . Weaver,qemu_bbv - a qemu patch that enables SimPoint Basic Block Vector File Generation, [Online] https://web.eece.maine.edu/~vweaver/ projects/qemusim
-
[16]
Gottschall, S
B. Gottschall, S. C. de Santana, M. Jahre,Balancing Accuracy and Evaluation Overhead in Simulation Point Selection, 2023 IEEE Interna- tional Symposium on Workload Characterization (IISWC), Ghent, Bel- gium, 2023, pp. 43-53, doi: 10.1109/IISWC59245.2023.00019. Available: https...
2023
-
[17]
R. E. Wunderlich, T. F. Wenisch, B. Falsafi, J. C. Hoe,SMARTS: ac- celerating microarchitecture simulation via rigorous statistical sampling, 30th Annual International Symposium on Computer Architecture, 2003. San Diego, CA, USA, 2003, pp. 84-95, doi: 10.1109/ISCA.2003.1206991...
2003 arXiv
-
[18]
Fang, et.al,NPS: A Framework for Accurate Program Sampling Using Graph Neural Network, arXiv:2304.08880
Y . Fang, et.al,NPS: A Framework for Accurate Program Sampling Using Graph Neural Network, arXiv:2304.08880. April 2023. Available: https: //arxiv.org/abs/2304.08880
2023 arXiv
-
[19]
Y . Luo, A. Joshi, A. Phansalkar, L. John, J. Ghosh,Analyzing and improving clustering based sampling for microprocessor simu- lation, 17th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD’05), Rio de Janeiro, Brazil,
-
[20]
Choukse, M
E. Choukse, M. Erez, A. Alameldeen,CompressPoints: An Evalua- tion Methodology for Compressed Memory Systems, IEEE Computer Architecture Letters, vol. 17, no. 2, pp. 126-129, 2018. Available: https://ieeexplore.ieee.org/document/8328832
2018
-
[21]
Marwan, M.C
N. Marwan, M.C. Romano, M. Thiel, J. Kurths,Recurrence plots for the analysis of complex systems, Physics Reports, V olume 438, Issues 5–6, January 2007, Pages 237-329, ISSN 0370-1573, arXiv:2501.13933. Available: https://doi.org/10.1016/j.physrep.2006.11.001 and https://doi. ...
-
[22]
Foote,Visualizing music and audio using self-similarity, In Proceed- ings of the seventh ACM international conference on Multimedia (Part 1) (MULTIMEDIA ’99)
J. Foote,Visualizing music and audio using self-similarity, In Proceed- ings of the seventh ACM international conference on Multimedia (Part 1) (MULTIMEDIA ’99). Association for Computing Machinery, New York, NY , USA, 77–80. Available: https://doi.org/10.1145/319463.319472
-
[23]
Müller, M
M. Müller, M. Clausen,Transposition-invariant self-similarity matrices. Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR 2007): 47–50. Available: https://www.researchgate.net/ publication/220723240_Transposition-Invariant_Self-Similarity_Matrices
2007
-
[24]
Junejo, E
I.N. Junejo, E. Dexter, I. Laptev, P. Pérez,Cross-View Action Recog- nition from Temporal Self-similarities. In: Computer Vision – ECCV
-
[2004]
Available: https://api.semanticscholar.org/CorpusID:17764788
-
[2005]
Available: https://ieeexplore.ieee
doi: 10.1109/CAHPC.2005.11. Available: https://ieeexplore.ieee. org/document/1592573
2005
-
[2008]
Springer, Berlin, Heidelberg
Lecture Notes in Computer Science, vol 5303. Springer, Berlin, Heidelberg. ISBN 978-3-540-88685-3. Available: https://doi.org/10.1007/ 978-3-540-88688-4_22
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.