Pith. sign in

REVIEW 4 major objections 4 minor 64 references

Tracing Optimization for Performance Modeling and Regression Detection

T0 review · 4 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper argues that tracing only the few performance-sensitive functions is enough to build accurate performance models and detect regressions, cutting overhead by more than 80% in most cases.

desk verdict Solid empirical extension of prior instrumentation-refinement work, with a useful systematic comparison of pruning criteria, but the production-readiness claim goes beyond what the same-generator evaluation can support. read the letter →

arxiv 2411.17548 v1 pith:XETEARE7 submitted 2024-11-26 cs.SE cs.PF

Kaveh Shahedi , Heng Li , Maxime Lamothe , Foutse Khomh This is my paper · ORCID
classification cs.SEcs.PF
keywords tracingoptimizationperformancemodelingregressiondetectionfunctionpruningShannonentropycoefficientofvariationblack-boxmodelsoverhead
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that most traced functions carry little performance information, so they can be omitted from tracing without hurting—and sometimes improving—the resulting performance models. Across five C/C++ benchmark programs, only 5.64% to 21.93% of functions were identified as performance-sensitive, and optimized models reached R² scores from 0.931 to 0.999, often matching or beating full-tracing baselines. Tracing overhead fell by more than 80% in most cases. The same optimized models detected injected performance regressions more reliably than directly comparing execution times, supporting use in production settings where tracing costs matter.

What carries the argument

The central mechanism is the classification of functions into performance-sensitive and performance-insensitive sets using dynamic statistical criteria (Shannon entropy, coefficient of variation, Spearman correlation removal, and linear-regression feature significance) and a static score (StaPerfSens). These criteria rank functions, and an automatic threshold from one-dimensional clustering (Ckmeans.1d.dp) plus derivative analysis splits them. The selected functions' call frequencies then feed into AutoML-tuned regressors (e.g., CatBoost, BayesianRidge), and model-prediction differences between program versions serve as regression detectors.

What would settle it

Run the optimized and full-tracing models on a workload from a different distribution than the training generator—e.g., production traffic logs—and check whether a pruned function dominates runtime while the optimized model's R² drops sharply relative to full tracing.

Watch

Extended reading notes

Core claim

The paper establishes that a small subset of functions—those whose execution time varies with input and contributes to the model—sufficiently explains program execution time. Using call frequencies of only these functions as features, black-box regressors predict execution time with high accuracy, and the pruned models sometimes outperform full-tracing models because removing correlated functions reduces noise. The paper further shows that comparing predictions from old and new version models, via the Mann–Whitney U test and Cliff's delta, can flag regressions even when direct execution-time comparison fails.

Load-bearing premise

The performance-sensitivity labels learned from 2,500 generated inputs carry over to new inputs and production workloads; if a pruned function becomes performance-critical under an unseen input or call path, the optimized model will miss the regression.

Editorial extensions

If this is right

  • Optimized tracing can run at commit level or in production, where full tracing is too expensive.
  • Model accuracy does not require more trace data; pruning can remove correlated noise and improve predictions.
  • The best pruning criterion is program-specific, so the framework must select the criterion and model type per program.
  • Regression detection works by statistically comparing the old and new versions' model predictions, catching regressions that raw execution time comparisons miss.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The sensitivity labels depend on the input distribution used during training; production workloads that exercise pruned functions through unseen call paths could invalidate the model, so periodic re-identification or coverage-guided input generation would be a natural safeguard.
  • The same pruning logic could extend beyond function calls to kernel events or I/O operations, potentially yielding similar overhead reductions in other tracing contexts.
  • Comparing model predictions between versions could be adapted into a continuous online anomaly detector using sliding windows rather than release-level snapshots.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a set of statistical pruning criteria (Shannon entropy, coefficient of variation, correlation-based redundancy removal, feature significance, a union criterion, and a static score) to identify performance-insensitive functions and exclude them from application-level tracing. Performance models are then built from call frequencies of the remaining functions to predict execution time. The evaluation on five C/C++ benchmark programs reports that only 5.64%–21.93% of functions need tracing, that tracing overhead is reduced by more than 80% in most cases, that optimized models reach R2 values up to 0.999 and sometimes outperform full-tracing models, and that model-based regression detection with an injected delay works better than directly comparing execution times.

Significance. If the empirical claims hold, the work is a practical contribution to performance modeling: it attacks a real cost (tracing overhead) and evaluates the payoff in both model accuracy and regression detection. The paper has concrete strengths: a replication package is announced, the study uses diverse benchmark programs of different sizes, optimized models are compared against a full-tracing baseline, and regression detection is compared against a direct execution-time baseline. It is also fair to say that the central modeling claim is not circular by construction: optimized models are tested on held-out inputs, not on the training inputs. However, the strength of the contribution is limited by the absence of uncertainty quantification, by model/criterion selection on the test data, and by the fact that the optimized tracing set is selected and evaluated on inputs from the same hand-built generator, leaving the production-readiness claim undemonstrated.

major comments (4)
  1. [§4.2–§4.3, Tables 3–4] The central quantitative claims are reported as single point estimates with no repeated runs, confidence intervals, or standard deviations. Execution-time overhead in Table 3 varies enormously (e.g., 77.1% to 2243.9% for full tracing), and the claim that tracing overhead is reduced by more than 80% in most cases is based on average values whose variability is unknown. I ask for repeated measurements or, at minimum, confidence intervals and statistical tests before the overhead-reduction claim can be assessed.
  2. [§3.1.2, §3.3.3, §5.3] The performance-sensitive functions are identified from 2,500 inputs generated by a manual, program-specific generator, and the optimized models are evaluated on 333 new inputs sampled from the same generator. Section 5.3 explicitly concedes that 'certain functions might exhibit uncovered behavior with unseen inputs.' This is precisely the load-bearing limitation: a production workload with a different input distribution can activate a pruned function through a different call path, and the optimized model would then have no feature for that function. The abstract’s production-readiness claim is therefore not supported by the current evidence. A concrete remedy is to evaluate on held-out input regimes, fuzzed inputs, or production traces, and to temper the production claim accordingly.
  3. [§3.3.1, §4.3, Table 4] For each program, the 'best' pruning criterion and the 'best' machine-learning model are selected from a large pool (10 criteria and multiple model types) using the same held-out test set. Reporting the best R2 after such selection, without nested cross-validation or a separate validation set, overstates the expected performance and can explain the cases where optimized models 'sometimes outperform' full tracing. Please specify the model-selection protocol more precisely and evaluate the final selected configuration on an independent test set.
  4. [§3.2.1–§3.2.3] The pipeline depends on several ad hoc thresholds: the mean ± 3*std outlier rule, the top-99th-percentile unique-call-frequency filter, the p<0.05 feature-significance cutoff, the Spearman ρ≥0.7 correlation-removal threshold, and the loess-derivative/Ckmeans change-point split. No sensitivity analysis is reported for these choices. Since the method is described as fully automated and the results are used to justify pruning, I ask for a sensitivity analysis over reasonable ranges of these parameters, or at least a discussion of which thresholds are load-bearing.
minor comments (4)
  1. [§4.1] Several cross-references appear to be wrong: 'see Section 3.4.3' for input generation should refer to Section 3.1.2, and the replication-package references to Section 3.4.3 should be corrected.
  2. [Table 2] The column layout of Table 2 is unclear in the submitted text, and some reported counts are inconsistent with the stated semantics: for example, the 'All' column for SU2 reports 9 functions while its constituent criteria (Entropy: 42, Performance Correlations: 48) are much larger. Please clarify the column definitions and verify the values.
  3. [Table 5] The optimized-model row for freqmine reports counts out of 14 versions while all other rows use 15; please explain the missing version or correct the denominator.
  4. [§5.1] The statement that overhead reduction 'remains consistent across all tools' is not supported by any experiment with alternative tracers; it should be removed or qualified.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the optimized-model claims are evaluated on held-out inputs, and the self-citations are methodological, not load-bearing.

full rationale

The paper's central derivation—selecting performance-sensitive functions via entropy, CoV, correlation, feature significance, or static scoring and then building models from the selected functions' call frequencies—is not circular. The function-selection step (Section 3.2.2) is performed on a 2,500-run analysis dataset, and the optimized models are trained and tested on a fresh 333-run dataset (Section 3.3.3), so the reported R2 values are held-out estimates rather than re-statements of the selection criterion. Feature Significance is a standard supervised feature-selection procedure (linear-regression p-values), and the final model is a different learner evaluated on new data; no equation is equivalent to its input. RQ3's regression detection compares models fitted to old and new versions; this is the intended mechanism, and the claimed ability to distinguish versions is an empirical result. The self-citations ([20], [43], [44]) are methodological or related-work references and are not load-bearing for the main claim: they do not supply a uniqueness theorem or an unverified premise on which the conclusion depends. The paper's own Section 5.3 limitation—'certain functions might exhibit uncovered behavior with unseen inputs'—is a legitimate external-validity threat about input-distribution shift, not a circularity. The practice of reporting the best pruning criterion and ML model per program could inflate reported accuracy through selection, but that is a statistical-validity concern, not a reduction of the prediction to its inputs. Overall circularity score: 1.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The central claim rests on the transferability of sensitivity labels, the validity of fluctuation metrics as proxies for model contribution, and several ad hoc thresholds. The paper provides no independent benchmark for the thresholds and acknowledges transferability as a threat in Section 5.3.

free parameters (6)
  • Outlier exclusion range = mean ± 3*std
    Section 3.2.1; chosen by convention, not justified for trace data, and applied before entropy/CoV computation.
  • Unique call frequency cutoff = top 99th percentile of program executions
    Section 3.2.1; functions with fewer unique call frequencies than this are removed; threshold is ad hoc.
  • Correlation removal threshold = Spearman rho >= 0.7
    Section 4.2; highly correlated functions are merged or removed; no sensitivity analysis is reported.
  • Feature significance p-value threshold = p < 0.05
    Section 3.2.2; conventional threshold, but used as a pruning criterion whose impact on final model accuracy is not analyzed across thresholds.
  • Injected regression delay = 5 microseconds
    Section 4.4; one fixed delay is used for all regression injections, and detection results may depend on this magnitude.
  • Threshold split via loess derivative = change point from first derivative plus Ckmeans.1d.dp
    Section 3.2.3; the automatic threshold is a tuned analytic choice, not a fitted value from an independent benchmark.
assumptions (6)
  • domain assumption Self-execution time fluctuations measured by entropy and CoV are valid proxies for a function's contribution to overall performance model accuracy.
    Section 3.2.2 defines performance-sensitive functions via these metrics; the paper does not independently validate that high-fluctuation functions are necessary or sufficient for accurate models.
  • domain assumption Performance-sensitivity labels learned from 2,500 generated inputs transfer to new inputs and production workloads.
    Section 3.1.2 and Section 5.3; construct validity admits certain functions may show uncovered behavior with unseen inputs.
  • domain assumption Call frequency features remain unaffected by performance regressions, so models trained on call frequencies can detect execution-time regressions.
    Section 3.1.4 states call frequencies are stable under regressions; this is a stated modeling premise.
  • ad hoc to paper The loess-smoothed first-derivative change point plus Ckmeans.1d.dp clustering yields a meaningful automatic threshold between sensitive and insensitive functions.
    Section 3.2.3; no ground truth for sensitivity and no sensitivity analysis for smoothing parameters.
  • standard math Mann-Whitney U and Cliff's delta applied to model predictions on new inputs correctly indicate regressions.
    Section 3.4.3; standard non-parametric tests, but assumes samples are independent and representative.
  • domain assumption uftrace with -pg/-finstrument-functions records accurate function-level self-times and call counts without systematic distortion of sensitivity rankings.
    Section 3.1.3; tracing overhead itself can perturb timings, and outlier removal may mask this.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Tracing Optimization for Performance Modeling and Regression Detection." pith.science (2026). https://pith.science/paper/XETEARE7

@misc{pith2026241117548,
  author       = {Pith},
  title        = {Pith review of: Tracing Optimization for Performance Modeling and Regression Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XETEARE7}},
  note         = {Machine review of arXiv:2411.17548}
}
read the original abstract

Software performance modeling plays a crucial role in developing and maintaining software systems. A performance model analytically describes the relationship between the performance of a system and its runtime activities. This process typically examines various aspects of a system's runtime behavior, such as the execution frequency of functions or methods, to forecast performance metrics like program execution time. By using performance models, developers can predict expected performance and thereby effectively identify and address unexpected performance regressions when actual performance deviates from the model's predictions. One common and precise method for capturing performance behavior is software tracing, which involves instrumenting the execution of a program, either at the kernel level (e.g., system calls) or application level (e.g., function calls). However, due to the nature of tracing, it can be highly resource-intensive, making it impractical for production environments where resources are limited. In this work, we propose statistical approaches to reduce tracing overhead by identifying and excluding performance-insensitive code regions, particularly application-level functions, from tracing while still building accurate performance models that can capture performance degradations. By selecting an optimal set of functions to be traced, we can construct optimized performance models that achieve an R-2 score of up to 99% and, sometimes, outperform full tracing models (models using non-optimized tracing data), while significantly reducing the tracing overhead by more than 80% in most cases. Our optimized performance models can also capture performance regressions in our studied programs effectively, demonstrating their usefulness in real-world scenarios. Our approach is fully automated, making it ready to be used in production environments with minimal human effort.

Figures

Figures reproduced from arXiv: 2411.17548 by the authors.

Figure 1
Figure 1. The overview of determining performance-sensitive functions, optimized performance modeling, and regression detection. [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. The CoV graph of the 631.deepsjeng program along with its smoothed curve using loess. 3.3 Performance Modeling Black-box machine learning performance models are used to predict a system’s performance metric using the system’s behavior without having full knowledge of it [22, 29, 43]. Our performance models aim to predict system performance (i.e., execution time) by leveraging trace data (i.e., call frequencies) of t… view at source ↗
Figure 3
Figure 3. The general overview of detecting performance regressions with the optimized performance model. [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

64 extracted references · 55 canonical work pages

  1. [1]

    [n. d.]. Gperftools. https://github.com/gperftools/gperftools

    work page

  2. [2]

    [n. d.]. Perf. https://perf.wiki.kernel.org/

    work page

  3. [3]

    loess - R Documentation - https://www.rdocumentation.org/packages/stat s/versions/3.6.2/topics/loess

    Accessed June 12, 2023. loess - R Documentation - https://www.rdocumentation.org/packages/stat s/versions/3.6.2/topics/loess

    work page 2023

  4. [4]

    Tarek M Ahmed, Cor-Paul Bezemer, Tse-Hsun Chen, Ahmed E Hassan, and Weiyi Shang. 2016. Studying the effectiveness of application performance management (apm) tools for detecting performance regressions for web applications: an experience report. In Proceedings of the 13th International Conference on Mining Software Repositories . 1–12

    work page 2016

  5. [5]

    Moez Ali. 2020. PyCaret: An open source, low-code machine learning library in Python . https://www.pycaret.org PyCaret version 1.0

    work page 2020

  6. [6]

    Dan Ardelean, Amer Diwan, and Chandra Erdman. 2018. Performance analysis of cloud applications. In 15th USENIX Symposium on Networked Systems Design and Implementation (NSDI 18) . 405–417

    work page 2018

  7. [7]

    Emre Ates, Lily Sturmann, Mert Toslali, Orran Krieger, Richard Megginson, Ayse K Coskun, and Raja R Sambasivan. 2019. An automated, cross-layer instrumentation framework for diagnosing performance problems in distributed applications. In Proceedings of the ACM Symposium on Cloud Computing. 165–170

    work page 2019

  8. [8]

    Cornel Barna, Marin Litoiu, and Hamoun Ghanbari. 2011. Autonomic load-testing framework. In Proceedings of the 8th ACM international conference on Autonomic computing. 91–100

    work page 2011

Show all 64 references
  1. [9]

    Moritz Beller, Hongyu Li, Vivek Nair, Vijayaraghavan Murali, Imad Ahmad, Jürgen Cito, Drew Carlson, Ari Aye, and Wes Dyer. 2023. Learning to learn to predict performance regressions in production at meta. In 2023 IEEE/ACM International Conference on Automation of Software Test...

    2023

  2. [10]

    Mohamed N Bennani and Daniel A Menasce. 2005. Resource allocation for autonomic data centers using analytic performance models. In Second international conference on autonomic computing (ICAC’05) . IEEE, 229–240

    2005

  3. [11]

    Christian Bienia, Sanjeev Kumar, Jaswinder Pal Singh, and Kai Li. 2008. The PARSEC benchmark suite: Characterization and architectural implications. In Proceedings of the 17th international conference on Parallel architectures and compilation techniques . 72–81

    2008

  4. [12]

    André B Bondi. 2015. Foundations of software and system performance engineering: process, performance modeling, requirements, testing, scalability, and practice. Pearson Education

    2015

  5. [13]

    Fabian Brosig, Nikolaus Huber, and Samuel Kounev. 2011. Automated extraction of architecture-level performance models of distributed component- based systems. In 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011) . IEEE, 183–192

    2011

  6. [14]

    Andreas Brunnert and Helmut Krcmar. 2017. Continuous performance evaluation and capacity planning using resource profiles for enterprise applications. Journal of Systems and Software 123 (2017), 239–262

    2017

  7. [15]

    Kistowski

    James Bucek, Klaus-Dieter Lange, and Jóakim v. Kistowski. 2018. SPEC CPU2017: Next-generation compute benchmark. In Companion of the 2018 ACM/SPEC International Conference on Performance Engineering . 41–42

    2018

  8. [16]

    Alexandru Calotoiu, Torsten Hoefler, Marius Poke, and Felix Wolf. 2013. Using automated performance modeling to find scalability bugs in complex codes. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis . 1–12

    2013

  9. [17]

    Zhuangbin Chen, Jinyang Liu, Yuxin Su, Hongyu Zhang, Xiao Ling, Yongqiang Yang, and Michael R Lyu. 2022. Adaptive performance anomaly detection for online service systems via pattern sketching. In Proceedings of the 44th International Conference on Software Engineering . 61–72

    2022

  10. [18]

    Norman Cliff. 2014. Ordinal methods for behavioral data analysis . Psychology Press

    2014

  11. [19]

    Cristian Coarfa, John Mellor-Crummey, Nathan Froyd, and Yuri Dotsenko. 2007. Scalability analysis of SPMD codes using expectations. InProceedings of the 21st annual international conference on Supercomputing . 13–22

    2007

  12. [20]

    Hetong Dai, Heng Li, Che-Shao Chen, Weiyi Shang, and Tse-Hsun Chen. 2020. Logram: Efficient Log Parsing Using𝑛 n-Gram Dictionaries. IEEE Transactions on Software Engineering 48, 3 (2020), 879–892

    2020

  13. [21]

    Mathieu Desnoyers and Michel R Dagenais. 2006. The lttng tracer: A low impact performance and behavior monitor for gnu/linux. In OLS (Ottawa Linux Symposium), Vol. 2006. Citeseer, 209–224

    2006

  14. [22]

    Diego Didona, Francesco Quaglia, Paolo Romano, and Ennio Torre. 2015. Enhancing performance prediction robustness by combining analytical modeling and machine learning. In Proceedings of the 6th ACM/SPEC international conference on performance engineering . 145–156

    2015

  15. [23]

    Thomas D Economon, Francisco Palacios, Sean R Copeland, Trent W Lukaczyk, and Juan J Alonso. 2016. SU2: An open-source suite for multiphysics simulation and design. Aiaa Journal 54, 3 (2016), 828–846

    2016

  16. [24]

    Naser Ezzati-Jivan, Houssem Daoud, and Michel R Dagenais. 2021. Debugging of performance degradation in distributed requests handling using multilevel trace analysis. Wireless Communications and Mobile Computing 2021 (2021), 1–17

    2021

  17. [25]

    Mostafa Farshchi, Jean-Guy Schneider, Ingo Weber, and John Grundy. 2015. Experience report: Anomaly detection of cloud application operations using log and cloud metric correlation analysis. In 2015 IEEE 26th international symposium on software reliability engineering (ISSRE) ...

    2015

  18. [26]

    Jay Fenlason and Richard Stallman. 1988. GNU gprof. GNU Binutils. A vailable online: http://www. gnu. org/software/binutils (accessed on 21 April 2018) (1988)

    1988

  19. [27]

    King Chun Foo, Zhen Ming Jiang, Bram Adams, Ahmed E Hassan, Ying Zou, and Parminder Flora. 2015. An industrial case study on the automated detection of performance regressions in heterogeneous environments. In 2015 IEEE/ACM 37th IEEE International Conference on Software Engine...

    2015

  20. [28]

    International Organization for Standardization (ISO). 2005. ISO/IEC 25000:2005, Software Engineering - Software Product Quality Requirements and Evaluation (SQuaRE)

    2005

  21. [29]

    Ruoyu Gao, Zhen Ming Jiang, Cornel Barna, and Marin Litoiu. 2016. A framework to evaluate the effectiveness of different load testing analysis techniques. In 2016 IEEE international conference on software testing, verification and validation (ICST) . IEEE, 22–32

    2016

  22. [30]

    Mohamad Gebai and Michel R Dagenais. 2018. Survey and analysis of kernel and userspace tracers on linux: Design, implementation, and overhead. ACM Computing Surveys (CSUR) 51, 2 (2018), 1–33

    2018

  23. [31]

    Zhenhuan Gong, Xiaohui Gu, and John Wilkes. 2010. Press: Predictive elastic resource scaling for cloud systems. In 2010 International Conference on Network and Service Management . Ieee, 9–16

    2010

  24. [32]

    John L Henning. 2006. SPEC CPU2006 benchmark descriptions. ACM SIGARCH Computer Architecture News 34, 4 (2006), 1–17

    2006

  25. [33]

    Lexiang Huang and Timothy Zhu. 2021. tprof: Performance profiling via structural aggregation and automated analysis of distributed systems traces. In Proceedings of the ACM Symposium on Cloud Computing . 76–91

    2021

  26. [34]

    Engin Ipek, Bronis R De Supinski, Martin Schulz, and Sally A McKee. 2005. An approach to performance prediction for parallel applications. In Euro-Par 2005 Parallel Processing: 11th International Euro-Par Conference, Lisbon, Portugal, August 30-September 2, 2005. Proceedings 1...

    2005

  27. [35]

    Tauseef Israr, Murray Woodside, and Greg Franks. 2007. Interaction tree algorithms to extract effective architecture and layered performance models from traces. Journal of Systems and Software 80, 4 (2007), 474–492

    2007

  28. [36]

    Zhen Ming Jiang and Ahmed E Hassan. 2015. A survey on load testing of large-scale software systems. IEEE Transactions on Software Engineering 41, 11 (2015), 1091–1118

    2015

  29. [37]

    Darren J Kerbyson, Henry J Alme, Adolfy Hoisie, Fabrizio Petrini, Harvey J Wasserman, and Mike Gittings. 2001. Predictive performance and scalability modeling of a large-scale application. In Proceedings of the 2001 ACM/IEEE conference on Supercomputing . 37–37

    2001

  30. [38]

    Mohammed Adib Khan and Naser Ezzati-Jivan. 2023. Multi-level Adaptive Execution Tracing for Efficient Performance Analysis. In 2023 IEEE/ACIS 21st International Conference on Software Engineering Research, Management and Applications (SERA) . IEEE, 104–109

    2023

  31. [39]

    Namhyung Kim. [n. d.]. uftrace: Function Graph Tracer for C/C++/Rust. https://github.com/namhyung/uftrace/

  32. [40]

    Naveen Kumar, Bruce R Childers, and Mary Lou Soffa. 2005. Low overhead program monitoring and profiling. ACM SIGSOFT Software Engineering Notes 31, 1 (2005), 28–34

    2005

  33. [41]

    Jan-Patrick Lehr, Alexandru Calotoiu, Christian Bischof, and Felix Wolf. 2019. Automatic instrumentation refinement for empirical performance modeling. In 2019 IEEE/ACM International Workshop on Programming and Performance Visualization Tools (ProTools) . IEEE, 40–47

    2019

  34. [42]

    Jan-Patrick Lehr, Alexander Hück, and Christian Bischof. 2018. PIRA: Performance instrumentation refinement automation. In Proceedings of the 5th ACM SIGPLAN International Workshop on Artificial Intelligence and Empirical Methods for Software Engineering and Parallel Computing...

    2018

  35. [43]

    Lizhi Liao, Jinfu Chen, Heng Li, Yi Zeng, Weiyi Shang, Jianmei Guo, Catalin Sporea, Andrei Toma, and Sarah Sajedi. 2020. Using black-box performance models to detect performance regressions under varying workloads: an empirical study. Empirical Software Engineering 25 (2020), ...

    2020

  36. [44]

    Lizhi Liao, Jinfu Chen, Heng Li, Yi Zeng, Weiyi Shang, Catalin Sporea, Andrei Toma, and Sarah Sajedi. 2021. Locating performance regression root causes in the field operations of web-based systems: An experience report. IEEE Transactions on Software Engineering 48, 12 (2021), ...

    2021

  37. [45]

    Jan Mußler, Daniel Lorenz, and Felix Wolf. 2011. Reducing the overhead of direct application instrumentation using prior static analysis. InEuropean Conference on Parallel Processing . Springer, 65–76

    2011

  38. [46]

    Nadim Nachar et al. 2008. The Mann-Whitney U: A test for assessing whether two independent samples come from the same distribution. Tutorials in quantitative Methods for Psychology 4, 1 (2008), 13–20

    2008

  39. [47]

    Thanh HD Nguyen, Bram Adams, Zhen Ming Jiang, Ahmed E Hassan, Mohamed Nasser, and Parminder Flora. 2012. Automated detection of performance regressions using statistical process control techniques. In Proceedings of the 3rd ACM/SPEC International Conference on Performance Engi...

    2012

  40. [48]

    Abad, José Nelson Amaral, Petr Tůma, and Alexandru Iosup

    Alessandro Vittorio Papadopoulos, Laurens Versluis, André Bauer, Nikolas Herbst, Jóakim von Kistowski, Ahmed Ali-Eldin, Cristina L. Abad, José Nelson Amaral, Petr Tůma, and Alexandru Iosup. 2021. Methodological Principles for Reproducible Performance Evaluation in Cloud Comput...

    2021

  41. [49]

    Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, and Andrey Gulin. 2018. CatBoost: unbiased boosting with categorical features. Advances in neural information processing systems 31 (2018)

    2018

  42. [50]

    David W Scott. 1979. On optimal and data-based histograms. Biometrika 66, 3 (1979), 605–610

    1979

  43. [51]

    Weiyi Shang, Ahmed E Hassan, Mohamed Nasser, and Parminder Flora. 2015. Automated detection of performance regressions using regression models on clustered performance counters. In Proceedings of the 6th ACM/SPEC International Conference on Performance Engineering . 15–26

    2015

  44. [52]

    Claude Elwood Shannon. 1948. A mathematical theory of communication. The Bell system technical journal 27, 3 (1948), 379–423. Manuscript submitted to ACM Tracing Optimization for Performance Modeling and Regression Detection 23

    1948

  45. [53]

    Samuel Sanford Shapiro and Martin B Wilk. 1965. An analysis of variance test for normality (complete samples). Biometrika 52, 3/4 (1965), 591–611

    1965

  46. [54]

    Connie U Smith. 2007. Introduction to software performance engineering: Origins and outstanding problems. Formal Methods for Performance Evaluation: 7th International School on Formal Methods for the Design of Computer, Communication, and Software Systems, SFM 2007, Bertinoro,...

    2007

  47. [55]

    Connie U Smith and Lloyd G Williams. 2002. Performance solutions: a practical guide to creating responsive, scalable software . Vol. 23. Addison-Wesley Reading

    2002

  48. [56]

    Charles Spearman. 1987. The proof and measurement of association between two things. The American journal of psychology 100, 3/4 (1987), 441–471

    1987

  49. [57]

    Haizhou Wang and Mingzhou Song. 2011. Ckmeans. 1d. dp: optimal k-means clustering in one dimension by dynamic programming. The R journal 3, 2 (2011), 29

    2011

  50. [58]

    Runan Wang, Giuliano Casale, and Antonio Filieri. 2022. Enhancing performance modeling of serverless functions via static analysis. InInternational Conference on Service-Oriented Computing . Springer, 71–88

    2022

  51. [59]

    Benjamin Welton and Barton P Miller. 2019. Diogenes: Looking for an honest cpu/gpu performance measurement tool. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis . 1–20

    2019

  52. [60]

    Pengcheng Xiong, Calton Pu, Xiaoyun Zhu, and Rean Griffith. 2013. vPerfGuard: an automated model-driven framework for application performance diagnosis in consolidated cloud environments. In Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering . 271–282

    2013

  53. [61]

    Junwen Yang, Pranav Subramaniam, Shan Lu, Cong Yan, and Alvin Cheung. 2018. How not to structure your database-backed web applications: a study of performance bugs in the wild. In Proceedings of the 40th International Conference on Software Engineering . 800–810

    2018

  54. [62]

    de Pádua, Weiyi Shang, Steve Sporea, Andrei Toma, and Sarah Sajedi

    Kundi Yao, Guilherme B. de Pádua, Weiyi Shang, Steve Sporea, Andrei Toma, and Sarah Sajedi. 2018. Log4perf: Suggesting logging locations for web-based systems’ performance monitoring. In Proceedings of the 2018 ACM/SPEC International Conference on Performance Engineering . 127–138

    2018

  55. [63]

    Jidong Zhai, Wenguang Chen, and Weimin Zheng. 2010. Phantom: predicting performance of parallel applications on large-scale parallel machines using a single node. ACM sigplan notices 45, 5 (2010), 305–314

    2010

  56. [64]

    Guoliang Zhao, Safwat Hassan, Ying Zou, Derek Truong, and Toby Corbin. 2021. Predicting performance anomalies in software systems at run-time. ACM Transactions on Software Engineering and Methodology (TOSEM) 30, 3 (2021), 1–33. Manuscript submitted to ACM

    2021

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.