Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

Memory Access Vectors: Improving Sampling Fidelity for CPU Performance Simulations

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Pith's one-line read By adding Memory Access Vectors to SimPoint's Basic Block Vectors, this paper lifts the projected-score correlation for 523.xalancbmk_r from 0.80 to 0.98 on a 192-core SoC.

desk verdict A plausible and genuinely new sampling vector, but the single benchmark result doesn't yet prove the method — worth referee time. read the letter →

arxiv 2506.02344 v1 pith:2E7SAM3E submitted 2025-06-03 cs.AR stat.AP

classification cs.ARstat.AP
keywords SimPointBasicBlockVectorsMemoryAccessperformanceprojectionphasedetectionSPECCPU2017523.xalancbmk_rsamplingfidelity
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

CPU architects project future-chip benchmark scores by simulating a small set of instruction windows picked by SimPoint, which groups windows by their Basic Block Vectors (BBVs). This paper argues that BBVs alone miss phase changes driven by memory access behavior, so workloads with array-indirect access patterns such as 523.xalancbmk_r get misrepresented samples and inaccurate projections. It introduces Memory Access Vectors (MAVs), functional histograms of accesses to 4096-byte memory regions, and feeds them into SimPoint's clustering alongside BBVs with a weighting that adapts to memory intensity. On a 192-core AmpereOne SoC, the projected-to-silicon correlation for 523.xalancbmk_r improves from 0.80 to 0.98, and from 0.84 to 0.95 on the 96-core part. The paper concludes that this combined sampling method can restore projection confidence for data-driven workloads.

What carries the argument

Memory Access Vector (MAV) is the central object: for each instruction window, count read and write accesses to unique 4096-byte physical-memory regions under functional execution, with no cache or TLB dependence. The processing pipeline then ranks regions by inverse access frequency so that rarely touched, miss-prone regions dominate, normalizes the whole MAV matrix by average row magnitude to preserve memory-intensity differences between phases, applies a 0.95 exponential decay over the previous ten windows for temporal locality, projects both BBV and MAV matrices to 15 dimensions with Gaussian random projection, and concatenates them into a 30-dimensional signature. The adaptive weighting scales the MAV contribution by the application-wide fraction of memory operations, so code-heavy applications stay BBV-dominated while data-heavy applications give memory behavior a strong vote. The combined matrix goes into SimPoint's k-means clustering to pick representative windows.

What would settle it

Run a complete (or much longer) simulation of 523.xalancbmk_r on the performance model and compare its SPECrate projection to AmpereOne silicon; if the full-trace model still lands near 0.80 correlation, the claimed improvement is an artifact of sample reweighting rather than a fidelity fix. A cheaper check is to apply BBV+MAV to another array-indirect benchmark and see whether the correlation gain over BBV-only sampling persists.

Watch

Extended reading notes

Core claim

The paper's central claim is that SimPoint's sampling error on 523.xalancbmk_r is a signature problem, not a model problem: BBVs encode which code runs but not which memory is touched, so recurring code with shifting data phases looks homogeneous. To test this, the authors collect MAV histograms for the benchmark's 980 billion instructions and build a 30-cluster SimPoint with BBV+MAV. BBV alone assigned two phases to the first 200 billion instructions (the Xerces parser region), while BBV+MAV assigns twelve, and the new phase boundaries line up with IPC variation measured on silicon. Re-running the standard projection flow with these new SimPoints raises the SPECrate correlation for 523.xalancbmk_r from 0.80 to 0.98 on the 192-core SoC and from 0.84 to 0.95 at 96 cores. The paper presents this as evidence that memory-access-aware sample selection, not a change to the performance model, closes the projection gap.

Load-bearing premise

The load-bearing premise is that the 20% projection gap for 523.xalancbmk_r comes from sample selection rather than from the performance model's own simulation error, since no full-trace run independently validates the model on that benchmark.

Editorial extensions

If this is right

  • For 523.xalancbmk_r, projected SPECrate correlation with AmpereOne silicon rises from 0.80 to 0.98 on the 192-core SoC and from 0.84 to 0.95 on the 96-core part.
  • The 30-cluster SimPoint configuration remains valid for BBV+MAV, so existing heuristics for choosing cluster counts carry over unchanged.
  • MAV collection adds only histogram-update work during the emulation-based preprocessing step; trace collection after SimPoint selection is unaffected.
  • The combined vectors split the Xerces parser region into 12 phases rather than 2, matching silicon IPC variation, which is why the old samples underrepresented it.
  • The authors expect the same approach to help other workloads with extensive array-indirect accesses, such as graph analytics and machine-learning inference.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension would apply BBV+MAV to 623.xalanc_s or to other SPEC CPU2017 rates with large data footprints; if the correlation gain does not reproduce there, the effect may be specific to 523.xalanc's parser/transformer split.
  • Because MAV uses absolute physical addresses at 4096-byte granularity, phase signatures may depend on memory layout and page allocation; hashing or virtualizing addresses before building histograms would test whether the improvement is layout-independent.
  • The matrix-level normalization and temporal-decay terms suggest the method could carry over to projections of multi-program or many-core SoCs, where memory pressure from co-running workloads is part of the phase signal.
  • If the method generalizes, published sampling studies that exclude 523.xalanc due to convergence problems could be repeated with MAV, letting the benchmark return to standard projection suites.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes Memory Access Vectors (MAV), a microarchitecture-independent representation of per-window memory access histograms, and integrates it with SimPoint's Basic Block Vectors (BBV) for phase detection. The combined BBV+MAV signature uses inverse-frequency sorting, matrix-level normalization, temporal decay, Gaussian random projection to 15 dimensions, and adaptive weighting by memory-op fraction. The authors apply this to 523.xalancbmk_r on an AmpereOne SoC and report that the projected-to-silicon ratio (which they call "correlation") improves from 0.80 to 0.98 at 192 cores and from 0.84 to 0.95 at 96 cores (Table II). They support this with recurrence plots, phase plots, and a silicon IPC overlay (Figures 1-4).

Significance. If the improvement is real and robust, the paper addresses a known gap: SimPoint-based sampling can fail on workloads with array-indirect memory accesses, and 523.xalancbmk_r has previously been excluded from evaluation due to convergence problems. The MAV idea is simple, microarchitecture-independent, and the implementation via QEMU is lightweight. The paper also explicitly motivates the problem for large server-class CPUs with many cores. However, the evidence is currently limited to a single benchmark and a single SoC configuration, with no sensitivity analysis for the many introduced hyperparameters and no direct sampling-only validation; the reported metric conflates sampling fidelity with performance-model bias. The contribution is potentially useful for the performance-evaluation community, but it is not yet established at the level claimed.

major comments (3)
  1. [Section IV-D, Table II] The central quantitative claim compares the performance model's projected SPECrate ratio against silicon, not sampling fidelity. The baseline 0.80 comes from the same model (Table I); if the model systematically mispredicts the Xerces parser region, shifting more samples into that region, as BBV+MAV does, could increase the projected score for reasons unrelated to how well the samples represent the full program. The paper lacks a sampling-only validation, such as comparing the weighted IPC or cache-miss rates of the selected samples against full-trace silicon measurements. The qualitative overlay in Figure 4 is suggestive but not quantified. Without such validation, the 80% to 98% improvement cannot be attributed specifically to improved sampling fidelity.
  2. [Section III, processing flow items 1-5] The MAV pipeline introduces several tunable hyperparameters with no sensitivity analysis: the 4096-byte bucket granularity, the inverse-frequency sorting transformation, the 0.95 decay factor over 10 windows, the 15-dimensional random projection, and the memory-op weighting fraction. The single reported improvement could reflect parameter selection tailored to 523.xalancbmk_r rather than a robust property of the method. The authors should report results across a range of these parameters (at minimum varying the cluster count and random seed for k-means) to demonstrate that the improvement is not an artifact of one configuration.
  3. [Table I and Table II, Section IV-D] The values labeled "correlation" in Tables I and II are actually per-benchmark ratios of projected score to silicon (e.g., 0.84, 0.80, 0.98), not correlation coefficients. This terminology is misleading: the abstract's "increasing it from 80% to 98%" describes a reduction of projection error from 20% to 2%, not a statistical correlation. The authors should either rename these column headers to "projection ratio" or "accuracy" or, if they intend true correlations across benchmarks, provide the appropriate computation and clarify the difference.
minor comments (6)
  1. [Section III, item 1] The description of inverse-frequency sorting is ambiguous for memory regions with zero accesses; the inverse of zero is undefined. The authors should specify how non-accessed regions are handled (e.g., excluded, or assigned a maximum value).
  2. [Section IV-A] The paper says MAV records accesses in the "physical address space" but later states it is "independent of microarchitectural caches or TLBs." Since QEMU typically operates on virtual addresses, the authors should clarify whether they use virtual or physical addresses and how this choice affects the representation.
  3. [Section IV-B, Figure 1] The recurrence plots lack axis labels and a color scale, making it difficult to interpret the distance measure used. Please specify what distance metric (Euclidean, Manhattan, etc.) is plotted and the range of values represented.
  4. [Abstract and throughout] The text frequently renders "MAV" with a space as "MA V"; please use a consistent notation throughout.
  5. [Section IV-B] The description of Figure 1 says "the left image" and "the center image" but does not clearly refer to the three panels by number; labeling the panels (a), (b), (c) would improve readability.
  6. [References] Several references contain unintended line breaks or spacing artifacts (e.g., "V olume", "doi.org/ 10.1145"). Please format the bibliography cleanly.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the claimed 0.80→0.98 improvement is an empirical comparison of two sampling methods against external silicon measurements, not a quantity forced by construction or by self-citation.

full rationale

The central result in Table II compares the projected SPECrate correlation for 523.xalancbmk_r on AmpereOne silicon under BBV-only and BBV+MAV sampling. This is an empirical measurement against an external target (silicon), and no equation defines MAV in terms of the target correlation or fits a parameter to the reported 98%. The method in Section III specifies a fixed processing flow (inverse-frequency sorting, normalization, temporal decay, random projection, adaptive weighting, k-means clustering) before presenting the result, and the improvement is not statistically forced by construction: MAV could plausibly have selected worse samples. The paper's references to AmpereOne product materials are silicon data sources rather than load-bearing self-citations, and the in-house simulator is validated against external benchmarks. The assumption that the performance model is accurate for 523 except for sampling is a modeling assumption, not a circular reduction. Thus no specific circular step can be exhibited, and the appropriate finding is no significant circularity.

Assumptions & free parameters 8 free parameters · 6 assumptions · 0 invented entities

The central result depends on hand-set numeric choices (10M instruction windows, 4KB buckets, 0.95 decay over 10 windows, 15-dimensional random projection, 30 clusters) and on the assumption that the in-house performance model is unbiased for 523.xalanc. None of these choices is ablated, and no artifact is shipped.

free parameters (8)
  • instruction window length = 10 million instructions
    Section IV-A sets the interval for BBV and MAV collection; changing the window changes every vector.
  • memory bucket granularity = 4096 bytes
    Section IV-A: chosen to align with common page size and practical runtime; no sensitivity analysis.
  • temporal decay factor = 0.95
    Section III step 3: applied as exponential decay over previous windows.
  • temporal history length = 10 instruction windows
    Section III step 3: the decay is applied to the previous 10 windows.
  • random projection dimension for MAV = 15
    Section III step 4: MAV is reduced to 15 dimensions to match BBV.
  • combined vector dimension = 30
    Section III step 4: BBV and MAV projections are concatenated.
  • SimPoint cluster count = 30
    Section IV-C: described as a common choice; no variation is tested.
  • memory-op weighting fraction = not stated
    Section III step 5: MAV is scaled by the application-wide percentage of memory operations, whose value for 523.xalanc is not reported.
assumptions (6)
  • standard math Gaussian Random Projection preserves the cluster structure at 15 dimensions.
    Section III.4 uses Johnson-Lindenstrauss-style projection but does not verify the reduced dimension for these matrices.
  • domain assumption Code-block frequency correlates with performance.
    Inherited from SimPoint [9] and applied to MAV similarity without independent validation.
  • ad hoc to paper The performance model is unbiased for 523.xalanc.
    The attribution of the 80% baseline error to sampling rather than model error is not tested.
  • ad hoc to paper Inverse-frequency sorting emphasizes performance-critical accesses.
    Section III.1 asserts rare regions are likely to cause misses, but no evidence is provided.
  • domain assumption SimPoint cluster-count heuristics remain valid after adding MAV.
    Section IV.C states no change to cluster selection is needed after adding MAV.
  • domain assumption QEMU functional memory addresses represent hardware-relevant memory behavior.
    MAV is collected in QEMU independent of caches and TLBs, but no validation shows guest addresses match the SoC's physical behavior.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Memory Access Vectors: Improving Sampling Fidelity for CPU Performance Simulations." pith.science (2026). https://pith.science/paper/2E7SAM3E

@misc{pith2026250602344,
  author       = {Pith},
  title        = {Pith review of: Memory Access Vectors: Improving Sampling Fidelity for CPU Performance Simulations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2E7SAM3E}},
  note         = {Machine review of arXiv:2506.02344}
}
read the original abstract

Accurate performance projection of large-scale benchmarks is essential for CPU architects to evaluate and optimize future processor designs. SimPoint sampling, which uses Basic Block Vectors (BBVs), is a widely adopted technique to reduce simulation time by selecting representative program phases. However, BBVs often fail to capture the behavior of applications with extensive array-indirect memory accesses, leading to inaccurate projections. In particular, the 523.xalancbmk_r benchmark exhibits complex data movement patterns that challenge traditional SimPoint methods. To address this, we propose enhancing SimPoint's BBV methodology by incorporating Memory Access Vectors (MAV), a microarchitecture independent technique that tracks functional memory access patterns. This combined approach significantly improves the projection accuracy of 523.xalancbmk_r on a 192-core system-on-chip, increasing it from 80% to 98%.

Figures

Figures reproduced from arXiv: 2506.02344 by the authors.

Figure 1
Figure 1. Self-Similarity plots of 523.xalancbmk_r showing BBV, MAV, and combined BBV+MAV. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. BBV-only phases and SimPoint selections for 523.xalanc. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. BBV+MAV phases and SimPoint selections for 523.xalanc. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: IPC plot of 523.xalanc on AmpereOne silicon. [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. CHIA: An open-source framework for principled, agentic AI-driven hardware/software co-design research

    cs.AR 2026-06 unverdicted novelty 7.0 of 10

    CHIA is an open-source framework for agentic AI-driven hardware/software co-design using CHIA loops as directed cyclic graphs, a tool library, and features for reliable experimentation, shown via five case studies.

Reference graph

Works this paper leans on

27 extracted references · 20 canonical work pages · cited by 1 Pith paper

  1. [1]

    [Online] https://amperecomputing.com/briefs/ampereone-family-product-brief

    AmpereOne® Product Brief (A192-32X), Ampere Computing. [Online] https://amperecomputing.com/briefs/ampereone-family-product-brief

  2. [2]

    Carlson, N

    D. Carlson, N. Simakov, R. Hadlich, A. Curtis, J. Martin, G. Verma, S. Chheda, F. Coskun, R. Gonzalez, D. Wood, F. Zhang, R. Harrison, E. Siegmann,The AmpereOne A192-32X in Perspective: Benchmarking a New Standard. In Proceedings of the 2025 International Conference on High Performance Computing in Asia-Pacific Region Workshops. Association for Computing ...

  3. [3]

    Bucek, K-D

    J. Bucek, K-D. Lange, J. Kistowski,SPEC CPU2017: Next-Generation Compute Benchmark, In Companion of the 2018 ACM/SPEC Interna- tional Conference on Performance Engineering (ICPE ’18). Association for Computing Machinery, New York, NY , USA. Available: https://doi. org/10.1145/3185768.3185771

  4. [4]

    Sherwood, E

    T. Sherwood, E. Perelman, G. Hamerly, B. Calder,Automatically charac- terizing large scale program behavior, ACM International Conference on on Architectural Support for Programming Languages and Operating Sys- tems, October 2002. Available: https://api.semanticscholar.org/CorpusID: 6445892

  5. [5]

    Hamerly, E

    G. Hamerly, E. Perelman, J. Lau, B. Calder,SimPoint 3.0: Faster and More Flexible Program Phase Analysis, Journal of Instruction- Level Parallelism, V olume 7, September 2005. Available: https://api. semanticscholar.org/CorpusID:11937761

  6. [6]

    Calder, T

    B. Calder, T. Sherwood, G. Hamerly, E. Perelman,SimPoint: Pick- ing Representative Samples to Guide Simulation, book chapter from Processor and System-on-Chip Simulationedited by R. Leupers and O. Temam, Springer ISBN 978-1-4419-6174-7, 2010. Available: https: //link.springer.com/book/10.1007/978-1-4419-6175-4

  7. [7]

    Singh and M

    S. Singh and M. Awasthi,Efficacy of Statistical Sampling on Con- temporary Workloads: The Case of SPEC CPU2017, IEEE Interna- tional Symposium on Workload Characterization (IISWC), Orlando, FL, USA, 2019, doi: 10.1109/IISWC47752.2019.9042114. Available: https: //ieeexplore.ieee.org/document/9042114

  8. [8]

    Colp, J.N

    E. Colp, J.N. Amaral, C. Benedicto, R.E. Rodrigues, E. Borin,Re- port: The 523.xalancbmk_r Benchmark, University of Alberta, Jan- uary 2018. [Online] Available: https://webdocs.cs.ualberta.ca/~amaral/ AlbertaWorkloadsForSPECCPU2017/reports/xalancbmk_report.html

Show all 27 references
  1. [9]

    J. Lau, J. Sampson, E. Perelman, G. Hamerly, B. Calder,The strong correlation between code signatures and performance, International Sym- posium on Performance Analysis of Systems and Software, March 2005. Available: https://ieeexplore.ieee.org/document/1430578

  2. [10]

    Singhal, et.al,Performance Analysis and Validation of the Intel ® Pentium ® 4 Processor on 90 nm Technology, Intel Technology Journal,

    R. Singhal, et.al,Performance Analysis and Validation of the Intel ® Pentium ® 4 Processor on 90 nm Technology, Intel Technology Journal,

  3. [11]

    Chilukuri, J

    A. Chilukuri, J. Milthorpe, B. Johnston,Characterizing Optimizations to Memory Access Patterns using Architecture-Independent Program Features, Proceedings of the International Workshop on OpenCL, 2020. Available: https://doi.org/10.48550/arXiv.2003.06064

  4. [12]

    J. Jang, H. Kim and H. Lee,Characterizing Memory Access Patterns of Various Convolutional Neural Networks for Utilizing Processing-in- Memory, 2023 International Conference on Electronics, Information, and Communication (ICEIC), Singapore, 2023. Available: https://doi.org/10. ...

  5. [13]

    Balaji, N

    V . Balaji, N. Crago, A. Jaleel and B. Lucia,P-OPT: Practical Optimal Cache Replacement for Graph Analytics, 2021 IEEE International Sym- posium on High-Performance Computer Architecture (HPCA), Seoul, Ko- rea (South), 2021. Available: https://doi.org/10.1109/HPCA51647.2021. 00062

  6. [14]

    V . Weaver,exp-bbv: Valgrind plugin that makes SimPoint Basic Block Vector Files, [Online] https://valgrind.org/docs/manual/bbv-manual.html and https://web.eece.maine.edu/~vweaver/projects/valsim

  7. [15]

    Weaver,qemu_bbv - a qemu patch that enables SimPoint Basic Block Vector File Generation, [Online] https://web.eece.maine.edu/~vweaver/ projects/qemusim

    V . Weaver,qemu_bbv - a qemu patch that enables SimPoint Basic Block Vector File Generation, [Online] https://web.eece.maine.edu/~vweaver/ projects/qemusim

  8. [16]

    Gottschall, S

    B. Gottschall, S. C. de Santana, M. Jahre,Balancing Accuracy and Evaluation Overhead in Simulation Point Selection, 2023 IEEE Interna- tional Symposium on Workload Characterization (IISWC), Ghent, Bel- gium, 2023, pp. 43-53, doi: 10.1109/IISWC59245.2023.00019. Available: https...

  9. [17]

    R. E. Wunderlich, T. F. Wenisch, B. Falsafi, J. C. Hoe,SMARTS: ac- celerating microarchitecture simulation via rigorous statistical sampling, 30th Annual International Symposium on Computer Architecture, 2003. San Diego, CA, USA, 2003, pp. 84-95, doi: 10.1109/ISCA.2003.1206991...

  10. [18]

    Fang, et.al,NPS: A Framework for Accurate Program Sampling Using Graph Neural Network, arXiv:2304.08880

    Y . Fang, et.al,NPS: A Framework for Accurate Program Sampling Using Graph Neural Network, arXiv:2304.08880. April 2023. Available: https: //arxiv.org/abs/2304.08880

  11. [19]

    Y . Luo, A. Joshi, A. Phansalkar, L. John, J. Ghosh,Analyzing and improving clustering based sampling for microprocessor simu- lation, 17th International Symposium on Computer Architecture and High Performance Computing (SBAC-PAD’05), Rio de Janeiro, Brazil,

  12. [20]

    Choukse, M

    E. Choukse, M. Erez, A. Alameldeen,CompressPoints: An Evalua- tion Methodology for Compressed Memory Systems, IEEE Computer Architecture Letters, vol. 17, no. 2, pp. 126-129, 2018. Available: https://ieeexplore.ieee.org/document/8328832

  13. [21]

    Marwan, M.C

    N. Marwan, M.C. Romano, M. Thiel, J. Kurths,Recurrence plots for the analysis of complex systems, Physics Reports, V olume 438, Issues 5–6, January 2007, Pages 237-329, ISSN 0370-1573, arXiv:2501.13933. Available: https://doi.org/10.1016/j.physrep.2006.11.001 and https://doi. ...

  14. [22]

    Foote,Visualizing music and audio using self-similarity, In Proceed- ings of the seventh ACM international conference on Multimedia (Part 1) (MULTIMEDIA ’99)

    J. Foote,Visualizing music and audio using self-similarity, In Proceed- ings of the seventh ACM international conference on Multimedia (Part 1) (MULTIMEDIA ’99). Association for Computing Machinery, New York, NY , USA, 77–80. Available: https://doi.org/10.1145/319463.319472

  15. [23]

    Müller, M

    M. Müller, M. Clausen,Transposition-invariant self-similarity matrices. Proceedings of the 8th International Conference on Music Information Retrieval (ISMIR 2007): 47–50. Available: https://www.researchgate.net/ publication/220723240_Transposition-Invariant_Self-Similarity_Matrices

  16. [24]

    Junejo, E

    I.N. Junejo, E. Dexter, I. Laptev, P. Pérez,Cross-View Action Recog- nition from Temporal Self-similarities. In: Computer Vision – ECCV

  17. [2004]

    Available: https://api.semanticscholar.org/CorpusID:17764788

  18. [2005]

    Available: https://ieeexplore.ieee

    doi: 10.1109/CAHPC.2005.11. Available: https://ieeexplore.ieee. org/document/1592573

  19. [2008]

    Springer, Berlin, Heidelberg

    Lecture Notes in Computer Science, vol 5303. Springer, Berlin, Heidelberg. ISBN 978-3-540-88685-3. Available: https://doi.org/10.1007/ 978-3-540-88688-4_22

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.