REVIEW 4 major objections 5 minor 27 references
Aneto: Predicting System Performance by Exploiting Cross-Workload Regularity
T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read Single-run counters suffice to recover the stall fraction of any workload, so CPI under any memory technology is predictable without simulation.
desk verdict Single-run blocking-factor prediction with a real 3x hardware check, but the 8x story still rests on an unmeasured constant-BF assumption. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The key object is the blocking factor $BF$, the fraction of an LLC miss's memory penalty that actually stalls the pipeline, introduced through the linear decomposition $CPI = CPI_0 + BF \cdot MPI \cdot MP$. Aneto's machinery is a cross-workload logit regression, fitted once per platform on a small reference corpus, which maps the log of the maximum memory stall per instruction ($MPI \cdot MP$) and the log of CPI onto $BF$; the CPI term disambiguates workloads with equal memory exposure but different latency-hiding ability. The prediction step subtracts the baseline miss penalty and adds the $BF$-weighted miss-penalty difference, so it never needs to compute $CPI_0$ explicitly. The machinery inherits two assumptions from the linear model: $BF$ and $MPI$ stay constant as the memory penalty changes, and the regression fitted on reference workloads generalizes to any new workload on the same machine.
What would settle it
Take a workload whose memory behavior is known to change with latency—for example, one that alternates between streaming and pointer-chasing phases, or whose prefetcher timeliness degrades when DRAM slows down—and run it at the baseline memory and at a target memory with about three times the penalty, measuring the actual CPI on both. Compute Aneto's prediction from the baseline counters and compare the CPI error against the paper's reported 12.7% median; if a substantial class of such workloads consistently shows errors well above the reported 90th percentile of 36%, the constant-blocking-factor regularity is not as general as claimed.
Extended reading notes
Core claim
On a fixed microarchitecture, the linear CPI decomposition $CPI = CPI_0 + BF \cdot MPI \cdot MP$ leaves the blocking factor $BF$ as the only quantity that must be inferred, and Aneto's discovery is that $BF$ is not arbitrary per workload: in the log-transformed plane of memory exposure ($MPI \cdot MP$) against CPI, workloads of very different suites arrange themselves along a smooth $BF$ gradient, so the same regression coefficients assign a blocking factor to a held-out workload from its single baseline point. The estimator is $\mathrm{logit}(BF) = \beta_0 + \beta_1 \log(MPI \cdot MP) + \beta_2 \log(CPI)$, with CPI doing the work of separating workloads that share the same memory exposure but hide latency differently. Substituting the predicted $BF$ into $CPI_B = CPI_A + BF \cdot MPI_A \cdot (MP_B - MP_A)$ yields first-order CPI prediction at any memory penalty without ever running the workload on the target memory. The paper's evidence is the regularity across 110 hardware workloads and 132 simulator traces, the direct DDR-to-HBM hardware validation at roughly three times the penalty with 12.7% median error, and the eight-times extrapolation where the model agrees with the reference linear model to within about 15% median on the newest tested AMD platform.
Load-bearing premise
The load-bearing premise is that a workload's stall fraction and its LLC miss rate do not change when the memory latency changes, so the blocking factor measured or inferred at one memory configuration is the right coefficient for any other configuration.
Editorial extensions
If this is right
- On a machine already fitted with the three-coefficient regression, predicting a new workload's CPI under any memory latency costs one five-minute run instead of a full frequency sweep or a simulation.
- The regression needs only a handful of reference workloads to fit: the paper measures the minimal corpus size at five to seven workloads, so fitting a new machine is a bounded one-time cost.
- Workload ordering by memory sensitivity is preserved, with a rank-correlation coefficient of 0.86 to 0.93 at three times the penalty, so the model can drive placement decisions such as which workloads belong in a fast HBM tier.
- The same equations apply in simulators, so design-space exploration can replace time-consuming re-simulation at each memory latency with a single baseline evaluation.
Reading between the lines
- If the regularity reflects a structural co-evolution of programs and hardware rather than an artifact of the benchmark mix, the same regression may transfer to new workload classes with only per-platform intercept shifts; this is testable by fitting on one processor generation and predicting blocking factors on the next.
- The model's success suggests a cheap screening layer before detailed cycle-level simulation: use a single-run blocking-factor estimate to shortlist latency-sensitive workloads, then simulate only those.
- Aneto's demand-side blocking-factor estimate composes naturally with a measured supply curve for the target memory, potentially predicting performance for memory technologies that do not physically exist yet, subject only to the supply curve's validity.
- A direct test of the mechanism would be to check whether $BF$ and $CPI_0$ are correlated because both trace back to one latent workload property; if so, the regression could be collapsed to one feature plus a platform offset.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Aneto, a cross-workload regression model that predicts the blocking factor (BF) of a workload from a single native run using three standard performance counters: CPI, LLC misses per instruction (MPI), and miss penalty (MP). The key equations are Eq. (2), logit(BF) = beta0 + beta1 log(MPI*MP) + beta2 log(CPI), and Eq. (4), CPI_B = CPI_A + BF_hat * MPI_A * (MP_B - MP_A). The model is fitted once per platform on a small corpus of reference workloads whose ground-truth BF is obtained from Clapp et al.'s multi-point linear regression, and is then applied to held-out workloads via leave-one-out cross-validation. The evaluation spans five x86 platforms, one ARM server, and three simulator configurations; the headline results are a median 12.7% CPI error on the ARM DDR-to-HBM validation at ~3x penalty, a roughly 2x error reduction relative to PROFET on ChampSim, and median 8x errors of 11.6-18.2% across platforms. The manuscript discloses the constant-BF assumption, the R^2 retention gate, and the model-vs-model nature of the hardware 8x extrapolation.
Significance. If the central claim holds, Aneto is a practically valuable tool: it reduces per-workload memory-latency sensitivity characterization from a multi-point frequency sweep or full simulation to a single three-counter measurement, while retaining the mechanistic CPI structure. The paper's strengths include a principled leave-one-out protocol, direct hardware validation on a real DDR-to-HBM system at a CXL-like penalty, honest disclosure of the model-vs-model nature of the 8x numbers, and a systematic comparison against PROFET using a paired statistical test. The main open risk is the constancy of BF and MPI under large memory-penalty changes; the manuscript discloses this as a limitation but does not provide the direct ChampSim-based BF-drift measurement that would close the gap.
major comments (4)
- [§3.5, §5.6, §5.8, Table 6] The correctness of the central prediction (Eq. 4) rests on BF_B = BF_A and MPI_B = MPI_A as MP varies, an assumption stated in §3.5 and acknowledged as the 'strongest limitation' in §5.8. The paper does not quantify how much of the reported error at 8x MP is due to drift in BF as opposed to error in the cross-workload regression. On ChampSim, direct simulation provides ground-truth CPI at up to 6.7-7.0x the baseline (Table 5), so the manuscript could compute, per workload, the BF implied at each simulated latency from CPI, MPI, and the baseline CPI0, and then report the distribution of BF_B - BF_A; this would directly test the load-bearing assumption. Without this, the 19% of high-memory-fraction workloads with errors above 25% at 3x (Fig. 9) are the cases where the assumption is most likely to break, and the cause is left unattributed.
- [§4.4.2, Table 6, Abstract] The 8x 'CPI error' columns in Table 6 are not errors against hardware measurements; §4.4.2 states that beyond the DVFS range both Aneto and Clapp et al. rely on the same linear CPI model, so the numbers are model-vs-model agreement. The table labels the comparison 'ground-truth BF', but that ground-truth BF is itself the slope of a linear fit that assumes BF is constant across the fit range. The abstract's phrase 'agrees with a reference model' is honest, but the body should consistently label these results as model-vs-model, and claims about 8x accuracy on real hardware should be based on the ChampSim results, where ground truth comes from direct simulation.
- [§4.2, §5.6, §3.5] The R^2 >= 0.90 retention filter is applied to the per-workload linear fit before computing LOO errors and is disclosed in §4.2 and §5.6, but it conditions the central claim. Because a practitioner applying Aneto to a new workload does not know its R^2 without performing the multi-point sweep the method is meant to avoid, the 'single run for any workload' claim in the abstract and §1 is stronger than what is evaluated. Please report the number of workloads excluded by the R^2 gate separately from those excluded by the MPI threshold (Table 3 gives only the post-filter counts), and provide a single-run observable diagnostic, such as the LOO residual or a confidence interval from the regression, that indicates when a new workload is likely to fall outside the model's validity.
- [§5.4, §4.2] The ARM DDR-to-HBM validation changes the execution configuration from the single-thread setup described in §4.2 to 16-copy rate runs in §5.4, but the paper does not state whether the Aneto coefficients and the 'single DDR run' used for prediction were obtained under the same 16-copy configuration or under the single-thread configuration. If the calibration used single-thread measurements and the prediction used rate measurements, the validation conflates a change in concurrency and bandwidth pressure with the memory-penalty change the model is designed to predict. Please specify the configuration for both the calibration sweep and the prediction run, and report the HBM errors for both configurations if possible.
minor comments (5)
- [§5.5] The values '3.2×10^-3 to 3.3×10^-3' for the Zen5 P90 are reported without units and are inconsistent with the percent-scale errors elsewhere; please state whether these are relative CPI errors, absolute BF errors, or something else.
- [§4.4.2, Table 5] The text says the DVFS sweep covers only 1.5-2.3x, but Table 5 reports MP factors of 0.4x-0.7x, meaning the most distant point lies below the baseline; clarify whether the 1.5-2.3x refers to the total MP spread and reconcile the notation.
- [Table 7] The W column in Table 7 (44 for all hardware platforms, 87 for all simulators) does not match the per-platform W counts in Table 3 (85/86/71/73/80/130/129/92); please explain which workload subset the subsampling and the 16-corpus comparison used, or recompute with the full qualifying sets.
- [§2, Step 3] The assertion that BPI_B = BPI_A and MPI_B = MPI_A 'we observe no changes over measurement noise' is not backed by data in the manuscript; include the supporting measurement or a reference to a figure.
- [§5.4, §5.8] There are several typos, including 'an prohibitive amount of resources' and 'the the predictive power'; a careful proofread is needed.
Circularity Check
No significant circularity: LOO cross-validation and independent hardware/simulator ground truth support the BF estimator.
full rationale
The derivation chain is self-contained rather than circular. Equation 2's training labels are Clapp's per-workload BF fits, and the LOO protocol holds out the target workload when predicting its BF from baseline counters, so the predicted BF is not the fitted label itself. Equation 4's CPI extrapolation is checked against measured hardware CPI within the DVFS range, direct ChampSim/Sniper simulation at up to ~7x penalty, and real ARM HBM measurements at ~3x penalty, all of which are independent of the fitted regression. The 8x results are explicitly disclosed as model-vs-model agreement in Section 4.4.2, which is a limitation rather than a hidden circular step. The R2>=0.90 filter and constant-BF/MPI assumptions are acknowledged scope conditions in Sections 3.5, 4.2, and 5.6, and there are no load-bearing self-citations. Remaining concerns about BF drift at large MP or possible in-sample ARM fitting are stated limitations or empirical risks, not reductions of the prediction to its inputs by construction.
Assumptions & free parameters
free parameters (5)
- beta_0 (per-platform intercept) =
not reported per platform
- beta_1 (log(MPI*MP) coefficient) =
0.50 (pooled AMD model)
- beta_2 (log(CPI) coefficient) =
0.42 (pooled AMD model)
- R^2 retention gate =
0.90
- MPI activity threshold =
1e-3
assumptions (4)
- domain assumption CPI = CPI0 + BF x MPI x MP, with BF and CPI0 constant as MP varies
- domain assumption MPI and BPI do not change with memory penalty
- domain assumption Ground-truth BF from Clapp's 5-point frequency sweep is an unbiased estimate of the true blocking factor
- domain assumption Hardware performance counters measure CPI, MPI, and MP accurately enough
Cite this review
Pith. "Pith review of Aneto: Predicting System Performance by Exploiting Cross-Workload Regularity." pith.science (2026). https://pith.science/paper/IMQZNLXD
@misc{pith2026260807179,
author = {Pith},
title = {Pith review of: Aneto: Predicting System Performance by Exploiting Cross-Workload Regularity},
year = {2026},
howpublished = {\url{https://pith.science/paper/IMQZNLXD}},
note = {Machine review of arXiv:2608.07179}
}
read the original abstract
Predicting how a workload responds to a change in memory technology requires estimating how much of each cache miss actually stalls the processor. Obtaining this stall fraction accurately has traditionally demanded detailed simulation, repeated measurements, or heavy profiling. One-shot alternatives exist but sacrifice accuracy. We observe that hardware counters from a single native run suffice to infer the stall fraction without simulation. Across more than 100 diverse workloads spanning integer, floating-point, graph, and AI benchmarks, the relationship between CPI and the maximum memory stall per instruction follows a predictable pattern on each microarchitecture. Aneto is a mechanistic-empirical regression model that exploits this observation. Once fitted on a machine across a small set of reference workloads, the model estimates the performance-latency sensitivity of any new workload from a single run, enabling first-order CPI prediction under any memory configuration. Across six machines and two simulators, Aneto reaches 2x lower CPI error than the best prior one-shot predictor. We validate the predictions directly against hardware measurements on an ARM server, from local DDR to HBM and up to ~3x the baseline memory penalty, where the median CPI error is 12.7% and the 90th percentile 35.9%. At an 8x memory-latency extrapolation beyond the reach of direct measurement, Aneto agrees with a reference model on Zen 5 to within 14.6% at the median and 41% at the 90th percentile. Additionally, Aneto provides qualitative insights into workloads and architectures.
Figures
Figures from the paper (6 more)
Reference graph
Works this paper leans on
-
[1]
Validation of the gem5 Simulator for x86 Architectures
Ayaz Akram and Lina Sawalha. “Validation of the gem5 Simulator for x86 Architectures”. In: 2019 IEEE/ACM Performance Modeling, Bench- marking and Simulation of High Performance Computer Systems (PMBS) . 2019, pp. 53–58. doi: 10.1109/PMBS49563.2019.00012
arXiv 2019
-
[2]
Scott Beamer, Krste Asanović, and David Patterson. “The GAP Benchmark Suite”. In: 2015. arXiv:1508. 03619
work page 2015
-
[3]
GraphMineSuite: Enabling High-Performance and Programmable Graph Mining Algorithms with Set Algebra
Maciej Besta, Zur Vonarburg-Shmaria, Yannick Schaffner, Leonardo Schwarz, Grzegorz Kwas- niewski, Lukas Gianinazzi, Jakub Beránek, Kacper Janda, Tobias Holenstein, Sebastian Leisinger, Peter Tatkowski, Esref Özdemir, Adrian Balla, Marcin Copik, Philipp Lindenberger, Marek Konieczny, Onur Mutlu, and Torsten Hoefler. “GraphMineSuite: Enabling High-Performan...
arXiv 2021
-
[4]
The PARSEC Benchmark Suite: Characterization and Architectural Implications
Christian Bienia, Sanjeev Kumar, Jaswinder Pal Singh, and Kai Li. “The PARSEC Benchmark Suite: Characterization and Architectural Implications”. In: Proc. PACT. 2008, pp. 72–81
work page 2008
-
[5]
Nathan Binkert, Bradford Beckmann, Gabriel Black, Steven K. Reinhardt, Ali Saidi, Arkaprava Basu, Joel Hestness, Derek R. Hower, Tushar Krishna, Somayeh Sardashti, Rathijit Sen, Korey Sewell, Muhammad Shoaib, Nilay Vaish, Mark D. Hill, and David A. Wood. “The gem5 simulator”. In:SIGARCH Comput. Archit. News 39.2 (Aug. 2011), pp. 1–7. issn: 0163-5964. doi:...
arXiv 2011
-
[6]
The DaCapo Benchmarks: Java Benchmarking Development and Analysis
Stephen M. Blackburn, Robin Garner, Chris Hoff- mann, Asjad M. Khang, Kathryn S. McKinley, Rotem Bentzur, Amer Diwan, Daniel Feinberg, Daniel Frampton, Samuel Z. Guyer, Martin Hirzel, Antony Hosking, Maria Jump, Han Lee, J. Eliot B. Moss, Aashish Phansalkar, Darko Stefanović, Thomas VanDrunen, Daniel von Dincklage, and Ben Wiedermann. “The DaCapo Benchmar...
work page 2006
-
[7]
SPEC CPU2017: Next-Generation Com- pute Benchmark
James Bucek, Klaus-Dieter Lange, and Jóakim von Kistowski. “SPEC CPU2017: Next-Generation Com- pute Benchmark”. In: Proc. ICPE Companion . 2018, pp. 41–42
work page 2018
-
[8]
Trevor E. Carlson, Wim Heirman, and Lieven Eeck- hout. “Sniper: Exploring the Level of Abstraction for Scalable and Accurate Parallel Multi-Core Simula- tion”. In:Proceedings of 2011 International Conference for High Performance Computing, Networking, Storage and Analysis (SC ’11) . ACM, 2011, 52:1–52:12. doi: 10.1145/2063384.2063454
arXiv 2011
Show all 27 references
-
[10]
A Simple Model to Quantify the Impact of Memory Latency and Bandwidth on Performance
Russell Clapp, Martin Dimitrov, Karthik Kumar, Vish Viswanathan, and Thomas Willhalm. “A Simple Model to Quantify the Impact of Memory Latency and Bandwidth on Performance”. In:Proceedings of the 2015 ACM SIGMETRICS International Conference on Measurement and Modeling of Compu...
2015
-
[11]
Quantifying the Performance Impact of Memory Latency and Bandwidth for Big Data Workloads
Russell Clapp, Martin Dimitrov, Karthik Kumar, Vish Viswanathan, and Thomas Willhalm. “Quantifying the Performance Impact of Memory Latency and Bandwidth for Big Data Workloads”. In:2015 IEEE In- ternational Symposium on Workload Characterization. 2015, pp. 213–224. doi:10.110...
2015 doi
-
[12]
Synthesis Lectures on Computer Architecture
Lieven Eeckhout.Computer Architecture Performance Evaluation Methods. Synthesis Lectures on Computer Architecture. Morgan & Claypool, 2010
2010
-
[13]
Understanding Some Simple Processor-Performance Limits
Philip G. Emma. “Understanding Some Simple Processor-Performance Limits”. In: IBM Journal of Research and Development 41.3 (1997), pp. 215–232
1997
-
[14]
A Mess of Memory System Benchmarking, Simulation and Application Profiling
Pouya Esmaili-Dokht, Francesco Sgherzi, Valéria Soldera Girelli, Isaac Boixaderas, Mariana Carmin, Alireza Monemi, Adrià Armejach, Estanislao Mer- cadal, Germán Llort, Petar Radojković, Miquel Moreto, Judit Giménez, Xavier Martorell, Eduard Ayguadé, Jesus Labarta, Emanuele Con...
2024
-
[15]
A mechanistic performance model for superscalar out-of-order processors
Stijn Eyerman, Lieven Eeckhout, Tejas Karkhanis, and James E. Smith. “A mechanistic performance model for superscalar out-of-order processors”. In: ACM Trans. Comput. Syst. 27.2 (May 2009). issn: 0734-2071. doi: 10 . 1145 / 1534909 . 1534910. uRl: https://doi.org/10.1145/15349...
2009
-
[16]
A Performance Counter Archi- tecture for Computing Accurate CPI Components
Stijn Eyerman, Lieven Eeckhout, Tejas Karkhanis, and James E. Smith. “A Performance Counter Archi- tecture for Computing Accurate CPI Components”. In: Proc. ASPLOS. 2006, pp. 175–184
2006
-
[17]
The Championship Simulator: Archi- tectural Simulation for Education and Competition
Nathan Gober,Gino Chacon, Lei Wang,Paul V.Gratz, Daniel A. Jiménez, Elvira Teran, Seth Pugsley, and Jinchun Kim. “The Championship Simulator: Archi- tectural Simulation for Education and Competition”. In: 2022. arXiv:2210.14324
2022 arXiv
-
[18]
Sources of error in full-system simulation
AnthonyGutierrez,JosephPusdesris,RonaldG.Dres- linski, Trevor Mudge, Chander Sudanthi, Christo- pher D. Emmons, Mitchell Hayenga, and Nigel Paver. “Sources of error in full-system simulation”. In: 2014 IEEE International Symposium on Performance Analysis of Systems and Softwar...
2014
-
[19]
Microarchitecture- Independent Workload Characterization
KennethHosteandLievenEeckhout.“Microarchitecture- Independent Workload Characterization”. In: IEEE Micro. Vol. 27. 3. 2007, pp. 63–72
2007
-
[20]
EfficientlyExploring ArchitecturalDesignSpacesviaPredictiveModeling
Engin Ipek, Sally A. McKee, Rich Caruana, Bronis R. deSupinski,andMartinSchulz.“EfficientlyExploring ArchitecturalDesignSpacesviaPredictiveModeling”. In: Proc. ASPLOS. 2006, pp. 195–206
2006
-
[21]
Construction and Use of Lin- ear Regression Models for Processor Performance Analysis
P. J. Joseph, Kapil Vaswani, and Matthew J. Thazhuthaveetil. “Construction and Use of Lin- ear Regression Models for Processor Performance Analysis”. In:Proc. HPCA. 2006, pp. 99–108
2006
-
[22]
A first-order superscalar processor model
T.S. Karkhanis and J.E. Smith. “A first-order superscalar processor model”. In: Proceedings. 31st Annual International Symposium on Com- puter Architecture, 2004. 2004, pp. 338–349. doi: 10.1109/ISCA.2004.1310786
2004 arXiv
-
[23]
A Day in the LifeofaDataCacheMiss
Tejas S. Karkhanis and James E. Smith. “A Day in the LifeofaDataCacheMiss”.In: Proc. Workshop on Mem- ory Performance Issues (WMPI). 2002
2002
-
[24]
Accurate and Efficient Regression Modeling for Microarchitectural PerformanceandPowerPrediction
Benjamin C. Lee and David M. Brooks. “Accurate and Efficient Regression Modeling for Microarchitectural PerformanceandPowerPrediction”.In: Proc. ASPLOS. 2006, pp. 185–194
2006
-
[25]
Concorde: Fast and Accurate CPU Per- formance Modeling with Compositional Analytical- ML Fusion
Arash Nasr-Esfahany, Mohammad Alizadeh, Victor Lee, Hanna Alam, Brett W. Coon, David E. Culler, VidushiDadu,MartinDixon,HenryM.Levy,Santosh Pandey, Parthasarathy Ranganathan, and Amir Yaz- danbakhsh. “Concorde: Fast and Accurate CPU Per- formance Modeling with Compositional An...
2025
-
[26]
PROFET: Modeling System Performance and Energy Without Simulating the CPU
Milan Radulovic, Rommel Sánchez Verdejo, Paul Carpenter, Petar Radojković, Bruce Jacob, and Eduard Ayguadé. “PROFET: Modeling System Performance and Energy Without Simulating the CPU”. In: Proc. ACM Meas. Anal. Comput. Syst. 3.2 (June 2019). doi: 10 . 1145 / 3341617 . 3326149....
2019
-
[27]
Ligra: A Light- weight Graph Processing Framework for Shared Memory
Julian Shun and Guy E. Blelloch. “Ligra: A Light- weight Graph Processing Framework for Shared Memory”. In:Proc. PPoPP. 2013, pp. 135–146
2013
-
[28]
An- alytical Processor Performance and Power Modeling Using Micro-Architecture Independent Characteris- tics
Sam Van den Steen, Stijn Eyerman, Sander De Pestel, Moncef Mechri, Trevor E. Carlson, David Black- Schaffer, Erik Hagersten, and Lieven Eeckhout. “An- alytical Processor Performance and Power Modeling Using Micro-Architecture Independent Characteris- tics”. In:IEEE Transaction...
2016
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.