Pith. sign in

REVIEW 4 major objections 5 minor 94 references

Enhancing Deployment-Time Predictive Model Robustness for Code Analysis and Optimization

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Prom identifies 96% of test inputs on which a deployed code-analysis or optimization model will mispredict, and updating the model on 5% of flagged inputs restores near-training-time performance.

desk verdict The detection half of this paper is solid and worth engaging; the deployment-recovery claim is not yet supported and needs a random-relabeling control and a specified retraining protocol. read the letter →

arxiv 2501.00298 v1 pith:H3KB64YP submitted 2024-12-31 cs.SE cs.AI

Huanting Wang , Patrick Lenihan , Zheng Wang This is my paper · ORCID
classification cs.SEcs.AI
keywords datadriftconformalpredictionmispredictiondetectiondeployment-timemodelrobustnessincrementallearningcodeoptimizationanalysismachinefor
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Prom is a deployment-time wrapper for already-trained machine-learning models used in code analysis and optimization. The paper's central claim is that Prom can flag, at inference time, the test inputs on which the deployed model will mispredict, catching on average 96% (up to 100%) of mispredictions with a false-positive rate below 14%. When a small fraction of flagged samples is relabeled and used to retrain the model, the paper reports that deployed performance returns to roughly the design-time level, with the performance-to-oracle ratio across 13 models and 5 tasks rising from 0.544 to 0.807. The practical payoff is that model developers need not anticipate every future hardware or workload change during training; the same trained model can be monitored and repaired in the field.

What carries the argument

The load-bearing mechanism is the adaptive conformal-prediction p-value. For each test input, Prom selects the closest 50% of calibration samples in the user-supplied feature space, exponentiates negative squared Euclidean distance to form a weight $w_i = \exp(-\|v_i-v_{n+1}\|^2/\tau)$ for each selected sample, multiplies each sample's nonconformity score by that weight, and computes the p-value as the fraction of same-label calibration samples whose weighted score meets or exceeds the test score. Four nonconformity functions vote on whether to reject each prediction, and a Gaussian-transformed prediction-set size serves as the confidence score. This machinery converts 'is this input like what the model saw in training?' into a thresholdable number without touching the underlying model, and the same p-value is extended to regression through k-NN pseudo-labels and K-means clustering of the calibration set.

What would settle it

Take a trained model and a drifted test set with known ground truth, run Prom at the default significance level, and compare the flagged set with the set of actual mispredictions: the paper's claim predicts at least 96% of true mispredictions flagged and fewer than 14% of correct predictions flagged. A concrete stress test is to construct a drifted workload whose control-flow structure changes but whose feature embedding is nearly unchanged (for example, permuting loop structure or renaming variables in an IR-based embedding); if such inputs are mispredicted yet not flagged, the Euclidean-distance premise fails.

Watch

Extended reading notes

Core claim

Prom operates by computing two statistical scores for each prediction during deployment: a credibility score, which measures how similar the test input is to calibration samples held out from training, and a confidence score, which measures how concentrated the model's probability output is on the predicted class. Both are derived from conformal prediction p-values, and Prom flags a prediction when both scores fall below a user-set significance level. Unlike earlier conformal drift detectors, Prom does not compare the test input against the full calibration set: it adaptively selects the nearest calibration samples in the model's feature space and weights their nonconformity scores by distance, then combines four nonconformity functions into an ensemble that votes on whether to reject the prediction. For regression, Prom approximates the unknown true label with a k-nearest-neighbour average and clusters the calibration set so that the same p-value machinery applies. The paper evaluates this on 13 independently developed models spanning thread coarsening, loop vectorization, heterogeneous device mapping, vulnerability detection, and DNN code generation, reporting that Prom identifies 96.2% of mispredictions on average and, after incremental learning on up to 5% of flagged samples, lifts the mean performance-to-oracle ratio from 0.544 to 0.807.

Load-bearing premise

The load-bearing premise is that a test input the model is about to mispredict will be far, in ordinary Euclidean distance, from the calibration samples in the numerical feature vectors the user supplies; if drift hides from that embedding, the flagging signal collapses.

Editorial extensions

If this is right

  • A deployed performance tuner can run an expensive fallback, such as autotuning or iterative compilation, only when Prom flags a prediction, so profiling effort is spent where the model is actually wrong.
  • A vulnerability detector can route flagged predictions to a human reviewer, turning expert attention into a targeted check rather than a full audit of every output.
  • Retraining on at most 5% of Prom-identified samples, sometimes a single sample, brings the deployed performance-to-oracle ratio from 0.544 back to 0.807 on average across the five tasks.
  • The same wrapper transfers across classical classifiers, LSTMs, transformers, and graph neural networks, because Prom only needs the model's probability output and a feature vector for each input.
  • Users can trade false positives against missed drift by adjusting the significance level, so a deployment with expensive rejection costs can choose a stricter threshold.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same adaptive-calibration machinery should transfer to deployment-time monitoring outside code, such as cloud resource prediction or medical image triage, since Prom only requires a probability output and a feature embedding; the paper does not test those domains.
  • Detection quality should depend on the embedding's sensitivity to drift: a testable extension is to measure Prom's recall on drift that changes program structure while leaving the feature vector nearly unchanged, which would stress the Euclidean-distance premise.
  • Because Eq. (2) weights calibration scores by distance, the exchangeability guarantee of split conformal prediction no longer applies; characterizing coverage under the adaptive weighting is an open question the paper addresses only empirically through coverage deviation.
  • The incremental-learning loop is currently a batch fine-tune; combining Prom with a fully online update of both the model and the calibration set is a natural next step the paper leaves for future work.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper introduces Prom, a model-agnostic framework that applies conformal prediction at deployment time to flag test inputs likely to be mispredicted by an already-trained classifier or regressor. Prom selects a local subset of calibration examples, weights their nonconformity scores by feature-space distance (Eq. 1), computes class-conditional p-values (Eq. 2), and combines four nonconformity functions by majority vote. The authors evaluate Prom on 13 ML models across five code analysis and optimization tasks, reporting an average misprediction recall of 96% (up to 100%) with a false-positive rate below 14%, and show that relabeling up to 5% of flagged samples with incremental learning raises the average deployment-to-oracle performance ratio from 0.544 to 0.807 (Table 2).

Significance. If the detection results are taken at face value, the paper makes a useful empirical contribution: it is the first application of conformal-prediction-style credibility scoring to code optimization workloads, and it does so in a model-agnostic way that supports both classification and regression. The evaluation is grounded in external ground truth (oracle searches, profiling time, CVE labels), the method is specified in enough detail to re-implement, and the authors provide an artifact and comparisons against MAPIE, PUNCC, RISE, and TESSERACT. The main weakness is that the second half of the headline claim, namely deployment-time recovery through incremental learning, is not yet supported by a controlled experiment. In addition, the deviation from exchangeability in Eq. (2) means the method's statistical validity is empirical rather than guaranteed, and the paper should present the supporting coverage data more transparently. The core premise that Euclidean distance in the user-supplied feature space tracks misprediction risk is an empirical assumption; the case-study breadth provides support but not a guarantee.

major comments (4)
  1. [Sec. 7.3, Table 2] The deployment-recovery claim is not yet established because the incremental-learning experiment has no random-relabeling control and no specified retraining protocol. As presented, the improvement from 0.544 to 0.807 could be obtained by adding any labeled samples from the deployment distribution, and the absence of epochs, learning rate, loss function, reuse of the original training set, and exclusion of relabeled samples from evaluation makes the result unreproducible. I request a random-relabeling baseline (and ideally an oracle-selection upper bound), a precise retraining specification, and per-case sample sizes and relabeling counts.
  2. [Sec. 8] The statement that "cross-validation shows that this retraining approach effectively improves generalization" is unsupported: no cross-validation result for the retrained models appears anywhere in the paper. This claim should be either removed or substantiated with the actual cross-validation numbers.
  3. [Sec. 5.1.2, Eq. (2)] The adaptive weighting and local subset selection used in Eq. (2) break the exchangeability assumption that underlies split conformal prediction, so the significance level 1−ε cannot be claimed as a validity guarantee for Prom's p-values. The reported geomean coverage deviation of 2.5% (Fig. 13(d)) is useful empirical evidence, but the text should state plainly that Eq. (2) is a heuristic and should report per-model and per-case coverage deviations rather than only the geomean, especially for the 4.4% thread-coarsening deviation caused by the small calibration set.
  4. [Sec. 7.4] Some of the headline detection percentages rest on very small test sets: case study 1 uses 17 OpenCL kernels, and case study 4's "recall of 1" is reported without the number of mispredictions in the deployment test set. The paper should report the number of test samples and the number of mispredictions per case study so that the 96% average and up-to-100% recall are interpretable with confidence intervals.
minor comments (5)
  1. [Sec. 3.3] The explanation of p-values is reversed: the text says a high p-value suggests the likelihood of the observed data under the null hypothesis is small, whereas a high p-value indicates that the data are likely under the null and provide weak evidence against it.
  2. [Sec. 5.3] The acceptance rule is stated ambiguously: the text says a sample is flagged if both scores "fall below the significance level," while earlier the significance level is defined as 1−ε; please clarify the direction of the comparison and the exact threshold used.
  3. [Table 2 caption] There is a typo in the caption: "Prom performence" should be "Prom performance."
  4. [Supplemental Material, Nonconformity functions] The introductory sentence contains an unresolved citation placeholder "[64? ? ? ]"; this should be fixed before publication.
  5. [Sec. 5.1.2] The notation for the number of selected calibration samples is inconsistent: the text refers to selecting "K samples" while Eq. (2) sums over i in {1,...,n}; please align the notation.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: central detection claim is validated against external ground truth; the incremental-learning gap is missing evidence, not a circular reduction.

full rationale

Prom's central claim—that its conformal scores flag inputs the underlying model will mispredict—is not derived from the labels it is asked to predict. The calibration p-values (Eq. 2) are functions of the model's own probability outputs and feature-space distances, and the reported recall/precision are computed against external ground truth (oracle search, profiling time, CVE labels) in Sec. 7.2. The regression pseudo-ground-truth in Sec. 5.1.1 is a k-NN average of calibration labels; this is a heuristic surrogate, but the evaluation still scores detection against actual profiling results, so it is not a case of the prediction being its own input. Sec. 7.3's incremental-learning recovery claim is weakened by the absence of a random-relabeling control and by an unspecified retraining/evaluation protocol, and Sec. 8's statement that 'cross-validation shows...' is not substantiated in the paper; these are reporting and validity gaps, not circular reductions. The only same-author citation used as prior art, RISE [83], is cited as inspiration and as a baseline, not as the proof of Prom's effectiveness. No load-bearing premise is justified solely by a self-citation, and no equation reduces to a fitted parameter renamed as a prediction. Hence no significant circularity.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

Prom introduces no new physical entity, mediator, or conserved quantity; the 'expert committee' is a descriptive label for the ensemble vote, not a new component. The audit therefore centers on free parameters and assumptions. Six numbers shape the results, the most consequential being the 20% misprediction thresholds, which define the very mispredictions that the 96% recall is measured against, and the hand-chosen tau, k, and selection fraction, none of which are swept. The assumptions are all domain-level or statistical-heuristic: the reweighted p-values are treated as calibrated without proof, and the feature-embedding distance is assumed to track misprediction risk. These do not render the work circular, but they do mean the headline numbers are conditional on the chosen feature spaces, thresholds, and defaults.

free parameters (6)
  • temperature tau in Eq. (1) = 500 (default)
    Scales distance weights for calibration samples; chosen by hand, sensitivity not reported.
  • nearest calibration fraction = 50% (or all samples if fewer than 200)
    Prunes the calibration set per test input; threshold chosen by hand, not swept.
  • k in k-NN pseudo-labeling and cluster assignment = 3
    Number of neighbors used to approximate regression ground truth and to assign cluster labels; no sensitivity analysis.
  • Gaussian scale c in confidence score = 3 (default)
    Maps prediction-set size to confidence; Fig. 13(c) shows sensitivity, default is a judgment call.
  • misprediction thresholds = 20% below oracle (cases 1-3) and 20% deviation from profiling (case 5)
    Metric-defining thresholds selected by the authors; changing them changes every reported detection and recovery number.
  • significance level epsilon = 0.1 (default)
    User-facing threshold; grid-search helper provided and sensitivity shown in Fig. 13(a).
assumptions (5)
  • ad hoc to paper Reweighted, locally selected calibration p-values (Eq. 1-2) remain approximately well-calibrated for flagging mispredictions
    The distance weighting and 50% nearest-neighbor pruning break split-CP exchangeability; no validity theorem is given, and the paper substitutes an empirical coverage-deviation check (Fig. 13(d)).
  • domain assumption Euclidean distance in the user-supplied feature embedding tracks misprediction risk under drift
    Invoked in Sec. 5.1.2 for calibration-subset selection and weighting; also the basis of k-NN pseudo-labeling in Sec. 5.1.1.
  • domain assumption Design-time calibration samples are sufficient to approximate a test sample's regression ground truth via k-NN averages
    Stated explicitly as a null hypothesis in Sec. 5.1.1; the detector flags samples where this approximation fails.
  • domain assumption K-means clusters of calibration feature vectors define meaningful classes for regression p-values
    Sec. 5.1.2 converts regression to pseudo-classification using Gap-statistic-selected clusters; cluster quality is not validated per task.
  • domain assumption Data drift, rather than model underfitting or label noise, is the operative cause of deployment-time degradation in the five tasks
    Sec. 2 motivates the whole approach with drift; the evaluation constructs train/test shifts to force drift.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing Deployment-Time Predictive Model Robustness for Code Analysis and Optimization." pith.science (2026). https://pith.science/paper/H3KB64YP

@misc{pith2026250100298,
  author       = {Pith},
  title        = {Pith review of: Enhancing Deployment-Time Predictive Model Robustness for Code Analysis and Optimization},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H3KB64YP}},
  note         = {Machine review of arXiv:2501.00298}
}
read the original abstract

Supervised machine learning techniques have shown promising results in code analysis and optimization problems. However, a learning-based solution can be brittle because minor changes in hardware or application workloads -- such as facing a new CPU architecture or code pattern -- may jeopardize decision accuracy, ultimately undermining model robustness. We introduce Prom, an open-source library to enhance the robustness and performance of predictive models against such changes during deployment. Prom achieves this by using statistical assessments to identify test samples prone to mispredictions and using feedback on these samples to improve a deployed model. We showcase Prom by applying it to 13 representative machine learning models across 5 code analysis and optimization tasks. Our extensive evaluation demonstrates that Prom can successfully identify an average of 96% (up to 100%) of mispredictions. By relabeling up to 5% of the Prom-identified samples through incremental learning, Prom can help a deployed model achieve a performance comparable to that attained during its model training phase.

Figures

Figures reproduced from arXiv: 2501.00298 by the authors.

Figure 1
Figure 1. (a) shows what happens if we train Vulde using CVE data collected between 2012 and 2014 and then apply it to real-life code samples developed between 2015 and 2023. This mimics a scenario where the code and bug patterns may evolve after a trained model is deployed. The trained model achieves an F1 score of more than 0.8 (ranging from 0 to 1, with higher being better) when the test and training data are collected fro… view at source ↗
Figure 2
Figure 2. Workflow of Prom during deployment. of probabilities needs to equal 1.0 [12]. In this case, a high probability does not equate to high prediction confidence. Therefore, assessing the model’s credibility requires an ap￾proach that evaluates how well the input aligns with the training data. 3.2 Statistical Assessment Prom uses statistical assessments to evaluate prediction cred￾ibility and confidence. Unlike typical p… view at source ↗
Figure 4
Figure 4. Simplified code template of Prom. 4 OVERVIEW of PROM [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (8 more)
Figure 5
Figure 5. Figure 5: Prom integrates multiple nonconformity functions that vote to reject or approve the ML prediction. method offers probabilistic values for 33 common ML models. In neural networks, probabilities can usually be extracted from the hidden layer before the output. For regres…
Figure 6
Figure 6. Figure 6: Prom dynamically selects a subset of the holdout calibration dataset to assess the test input’s nonconformity. 5 METHODOLOGY As shown in [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 7
Figure 7. Figure 7: The resulting performance when using an ML model for decision making. The performance of all learning-based models can suffer during the deployment phase when the test samples significantly differ from the training data. AccuracyPrecision Recall F1 Score 0.8 0.85 0.9 0…
Figure 8
Figure 8. Figure 8: Prom’s performance for detecting drifting samples across case studies and underlying models (higher is better). are marked as circles. Ideally, a model’s violin would be wide at the top, reflecting good performance for most samples. Design time performance. For case st…
Figure 9
Figure 9. Figure 9: Prom enhances performance through incremental learning in different underlying models. Case 1 Case 2 Case 3 Case 4 0.2 0.4 0.6 0.8 1 F1 Score RISE TESSERACT MAPIE-PUNCC PROM [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 11
Figure 11. Figure 11: Performance of individual nonconformity functions. Min-max bar shows the performance across ML models. 0 6 12 18 24 30 Overhead (hours) C5 C2 C4 C3 C1 Case study Initial Training Incremental Learning [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]
Figure 12
Figure 12. Figure 12: The average training and incremental learning overhead of individual case studies. 0 0.2 0.4 0.6 0.8 1 Significant level 0.9 0.95 1 Metric value Precision Recall F1 score (a) Prom performance as the threshold increases in loop vector￾ization 5 10 15 20 25 30 Cluster s…
Figure 13
Figure 13. Figure 13: Sensitive analysis of Prom hyperparameters. drifting samples across case studies, with min-max bars in￾dicating variance across models. Prom’s ensemble strategy outperforms individual functions in all metrics, demonstrat￾ing that no single function performs well acros…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

94 extracted references · 63 canonical work pages

  1. [1]

    [n. d.]. PROM: Enhancing Deployment-Time Predictive Model Robust- ness for Code Analysis and Optimization - Research Paper Artifact. https://doi.org/10.5281/zenodo.14077780

    work page doi:10.5281/zenodo.14077780

  2. [2]

    [n. d.]. pybind11. https://github.com/pybind/pybind11

  3. [3]

    Samuel Ackerman, Orna Raz, Marcel Zalmanovici, and Aviad Zlotnick

  4. [4]

    RR Ade and PR Deshmukh. 2013. Methods for incremental learning: a survey. International Journal of Data Mining & Knowledge Management Process (2013)

    2013

  5. [5]

    Anastasios N Angelopoulos and Stephen Bates. 2021. A gentle in- troduction to conformal prediction and distribution-free uncertainty quantification. arXiv preprint arXiv:2107.07511 (2021)

    arXiv 2021

  6. [6]

    Anastasios Nikolas Angelopoulos, Stephen Bates, Michael Jordan, and Jitendra Malik. 2020. Uncertainty Sets for Image Classifiers using Conformal Prediction. In International Conference on Learning Repre- sentations

    2020

  7. [7]

    Jason Ansel, Shoaib Kamil, Kalyan Veeramachaneni, Jonathan Ragan- Kelley, Jeffrey Bosboom, Una-May O’Reilly, and Saman Amarasinghe

  8. [8]

    Daniel Arp, Erwin Quiring, Feargus Pendlebury, Alexander Warnecke, Fabio Pierazzi, Christian Wressnegger, Lorenzo Cavallaro, and Konrad Rieck. 2022. Dos and don’ts of machine learning in computer security. In 31st USENIX Security Symposium (USENIX Security 22) . 3971–3988

    2022

Show all 94 references
  1. [9]

    David Arthur and Sergei Vassilvitskii. 2007. K-means++ the advantages of careful seeding. In Proceedings of the eighteenth annual ACM-SIAM symposium on Discrete algorithms . 1027–1035

    2007

  2. [10]

    Vineeth Balasubramanian, Shen-Shyang Ho, and Vladimir Vovk. 2014. Conformal prediction for reliable machine learning: theory, adaptations and applications. Newnes

    2014

  3. [11]

    Federico Barbero, Feargus Pendlebury, Fabio Pierazzi, and Lorenzo Cavallaro. 2020. Transcending transcend: Revisiting malware classifi- cation with conformal evaluation. arXiv (2020)

    2020

  4. [12]

    Christopher M Bishop. 2006. Pattern recognition and machine learning . Springer

    2006

  5. [13]

    Alexander Brauckmann, Andrés Goens, Sebastian Ertel, and Jeronimo Castrillon. 2020. Compiler-based graph representations for deep learn- ing models of code. In Proceedings of the 29th International Conference on Compiler Construction. 201–211

    2020

  6. [14]

    Wieland Brendel, Jonas Rauber, and Matthias Bethge. 2018. Decision- Based Adversarial Attacks: Reliable Attacks Against Black-Box Ma- chine Learning Models. In International Conference on Learning Repre- sentations

    2018

  7. [15]

    Ferhat Ozgur Catak, Javed Ahmed, Kevser Sahinbas, and Zahid Hussain Khand. 2021. Data augmentation based malware detection using convolutional neural networks. Peerj computer science 7 (2021), e346

    2021

  8. [16]

    Tianqi Chen, Thierry Moreau, Ziheng Jiang, Lianmin Zheng, Eddie Yan, Haichen Shen, Meghan Cowan, Leyuan Wang, Yuwei Hu, Luis Ceze, et al. 2018. {TVM}: An automated{End-to-End} optimizing compiler for deep learning. In 13th USENIX Symposium on Operating Systems Design and Imple...

    2018

  9. [17]

    Giovanni Cherubin, Konstantinos Chatzikokolakis, and Martin Jaggi

  10. [18]

    Chris Cummins, Zacharias V Fisches, Tal Ben-Nun, Torsten Hoefler, Michael FP O’Boyle, and Hugh Leather. 2021. Programl: A graph- based program representation for data flow analysis and compiler optimizations. In International Conference on Machine Learning . PMLR, 2244–2253

    2021

  11. [20]

    In International Conference on Machine Learning

    Exact optimization of conformal predictors via incremental and decremental learning. In International Conference on Machine Learning. PMLR, 1836–1845

  12. [21]

    Per-Erik Danielsson. 1980. Euclidean distance mapping. Computer Graphics and image processing 14, 3 (1980), 227–248

    1980

  13. [22]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova

  14. [23]

    Annette J Dobson and Adrian G Barnett. 2018. An introduction to generalized linear models. CRC press

    2018

  15. [24]

    Chris Cummins, Pavlos Petoumenos, Zheng Wang, and Hugh Leather

  16. [25]

    In 2017 IEEE/ACM International Symposium on Code Generation and Optimiza- tion (CGO)

    Synthesizing benchmarks for predictive modeling. In 2017 IEEE/ACM International Symposium on Code Generation and Optimiza- tion (CGO). IEEE, 86–99. Enhancing Deployment-Time Predictive Model Robustness for Code Analysis and Optimization CGO ’25, March 01–05, 2025, Las Vegas, NV, USA

    2017

  17. [26]

    Common Weakness Enumeration. 2023. 2023 CWE Top 25 Most Dangerous Software Errors. https://cwe.mitre.org/top25/archive/2023/2023_top25_list.html

    2023

  18. [27]

    Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, Ting Liu, Daxin Jiang, and Ming Zhou. 2020. CodeBERT: A Pre-Trained Model for Programming and Natural Languages. In Findings of the Association for Computational Linguistics: EMNLP ...

    2020 doi

  19. [28]

    Michael Fu and Chakkrit Tantithamthavorn. 2022. Linevul: A transformer-based line-level vulnerability prediction. In Proceedings of the 19th International Conference on Mining Software Repositories . 608–620

    2022

  20. [29]

    Jakob Gawlikowski, Cedrique Rovile Njieutcheu Tassi, Mohsin Ali, Jongseok Lee, Matthias Humt, Jianxiang Feng, Anna Kruspe, Rudolph Triebel, Peter Jung, Ribana Roscher, et al. 2023. A survey of uncertainty in deep neural networks.Artificial Intelligence Review56, Suppl 1 (2023)...

    2023

  21. [30]

    Sonia Domínguez-Almendros, Nicolás Benítez-Parejo, and Amanda Rocío Gonzalez-Ramirez. 2011. Logistic regression models. Allergologia et immunopathologia 39, 5 (2011), 295–305

    2011

  22. [31]

    Murali Krishna Emani and Michael F. P. O’Boyle. 2015. Celebrating di- versity: a mixture of experts approach for runtime mapping in dynamic environments. In Proceedings of the 36th ACM SIGPLAN Conference on Programming Language Design and Implementation, Portland, OR, USA, Jun...

    2015

  23. [32]

    Dominik Grewe, Zheng Wang, and Michael FP O’Boyle. 2013. Portable mapping of data parallel programs to opencl for heterogeneous sys- tems. In Proceedings of the 2013 IEEE/ACM International Symposium on Code Generation and Optimization (CGO) . IEEE, 1–10

    2013

  24. [33]

    Antonio Guerriero, Roberto Pietrantuono, and Stefano Russo. 2021. Operation is the hardest teacher: estimating DNN accuracy looking for mispredictions. In 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE). IEEE, 348–358

    2021

  25. [34]

    Ameer Haj-Ali, Nesreen K Ahmed, Ted Willke, Yakun Sophia Shao, Krste Asanovic, and Ion Stoica. 2020. NeuroVectorizer: end-to-end vectorization with deep reinforcement learning. In Proceedings of the 18th ACM/IEEE International Symposium on Code Generation and Opti- mization. 242–255

    2020

  26. [35]

    Masayo Haneda, Peter MW Knijnenburg, and Harry AG Wijshoff

  27. [36]

    Alexander Gepperth and Barbara Hammer. 2016. Incremental learning algorithms and applications. In ESANN

    2016

  28. [37]

    Andrés Goens, Alexander Brauckmann, Sebastian Ertel, Chris Cum- mins, Hugh Leather, and Jerónimo Castrillón. 2019. A case study on machine learning for synthesizing benchmarks. In Proceedings of the 3rd ACM SIGPLAN International Workshop on Machine Learning and Programming Lan...

    2019

  29. [38]

    Toru Kisuki, Peter MW Knijnenburg, and Michael FP O’Boyle. 2000. Combined selection of tile sizes and unroll factors using iterative compilation. In Proceedings 2000 International Conference on Parallel Architectures and Compilation Techniques (Cat. No. PR00622) . IEEE, 237–246

    2000

  30. [39]

    Zhen Li, Deqing Zou, Shouhuai Xu, Xinyu Ou, Hai Jin, Sujuan Wang, Zhijun Deng, and Yuyi Zhong. 2018. VulDeePecker: A Deep Learning- Based System for Vulnerability Detection. Proceedings of the NDSS (2018)

    2018

  31. [40]

    Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin Clement, Dawn Drain, Daxin Jiang, Duyu Tang, et al . 2021. CodeXGLUE: A Machine Learning Benchmark Dataset for Code Understanding and Generation. InThirty-fifth Confer- ence on Neural Infor...

    2021

  32. [41]

    Alberto Magni, Christophe Dubach, and Michael O’Boyle. 2014. Auto- matic optimization of thread-coarsening for graphics processors. In Proceedings of the 23rd international conference on Parallel architectures and compilation techniques. 455–466

    2014

  33. [42]

    Ankur Mallick, Kevin Hsieh, Behnaz Arzani, and Gauri Joshi. 2022. Matchmaker: Data drift mitigation in machine learning for large-scale systems. Proceedings of Machine Learning and Systems 4 (2022), 77–94

    2022

  34. [43]

    Vladislav Ishimtsev, Alexander Bernstein, Evgeny Burnaev, and Ivan Nazarov. 2017. Conformal𝑘-NN Anomaly Detector for Univariate Data Streams. In Conformal and Probabilistic Prediction and Applications . PMLR, 213–227

    2017

  35. [44]

    Roberto Jordaney, Kumar Sharad, Kumar Dash Santanu, Zhi Wang, Davide Papini, Ilia Nouretdinov, and Lorenzo Cavallaro. 2017. Tran- scend: Detecting Concept Drift in Malware Classification Models. In USENIX Security

    2017

  36. [45]

    Alexey Natekin and Alois Knoll. 2013. Gradient boosting machines, a tutorial. Frontiers in neurorobotics 7 (2013), 21

    2013

  37. [46]

    Ricardo Nobre, Luiz GA Martins, and João MP Cardoso. 2016. A graph- based iterative compiler pass selection and phase ordering approach. ACM SIGPLAN Notices 51, 5 (2016), 21–30

    2016

  38. [47]

    Dorit Nuzman, Ira Rosen, and Ayal Zaks. 2006. Auto-vectorization of interleaved data for SIMD. ACM SIGPLAN Notices 41, 6 (2006), 132–143

    2006

  39. [48]

    William F Ogilvie, Pavlos Petoumenos, Zheng Wang, and Hugh Leather

  40. [49]

    Feargus Pendlebury, Fabio Pierazzi, Roberto Jordaney, Johannes Kinder, and Lorenzo Cavallaro. 2019. TESSERACT: Eliminating Experimental Bias in Malware Classification across Space and Time. In USENIX Security

    2019

  41. [50]

    Vicent Sanz Marco, Ben Taylor, Barry Porter, and Zheng Wang. 2017. Improving Spark Application Throughput via Memory Aware Task Co-Location: A Mixture of Experts Approach. In Proceedings of the 18th ACM/IFIP/USENIX Middleware Conference (Las Vegas, Nevada) (Middleware ’17). As...

    2017

  42. [51]

    Mouhcine Mendil, Luca Mossina, and David Vigouroux. 2023. PUNCC: a Python Library for Predictive Uncertainty Calibration and Confor- malization. In Conformal and Probabilistic Prediction with Applications . PMLR, 582–601

    2023

  43. [52]

    Jie Ren, Ling Gao, Hai Wang, and Zheng Wang. 2017. Optimise web browsing on heterogeneous mobile platforms: a machine learning based approach. In IEEE INFOCOM 2017-IEEE Conference on Computer Communications. IEEE, 1–9

    2017

  44. [53]

    Jie Ren, Ling Gao, Xiaoming Wang, Miao Ma, Guoyong Qiu, Hai Wang, Jie Zheng, and Zheng Wang. 2021. Adaptive computation offloading for mobile augmented reality. Proceedings of the ACM on Interactive, CGO ’25, March 01–05, 2025, Las Vegas, NV, USA Huanting Wang, Patrick Lenihan...

    2021

  45. [54]

    Mauro Ribeiro, Katarina Grolinger, and Miriam AM Capretz. 2015. Mlaas: Machine learning as a service. In 2015 IEEE 14th international conference on machine learning and applications (ICMLA) . IEEE, 896– 902

    2015

  46. [55]

    Irina Rish et al. 2001. An empirical study of the naive Bayes classifier. In IJCAI 2001 workshop on empirical methods in artificial intelligence , Vol. 3. 41–46

    2001

  47. [56]

    In 2017 IEEE/ACM International Symposium on Code Generation and Optimization (CGO)

    Minimizing the cost of iterative compilation with active learning. In 2017 IEEE/ACM International Symposium on Code Generation and Optimization (CGO). IEEE, 245–256

    2017

  48. [57]

    Yaniv Romano, Matteo Sesia, and Emmanuel Candes. 2020. Classifica- tion with valid and adaptive coverage. Advances in Neural Information Processing Systems 33 (2020), 3581–3591

    2020

  49. [58]

    Pimentel, David A

    Marco A.F. Pimentel, David A. Clifton, Lei Clifton, and Lionel Tarassenko. 2014. A review of novelty detection. Signal Processing 99 (2014), 215–249. https://doi.org/10.1016/j.sigpro.2013.12.026

    2014 doi

  50. [59]

    Joaquin Quinonero-Candela, Masashi Sugiyama, Anton Schwaighofer, and Neil D Lawrence. 2008. Dataset shift in machine learning . Mit Press

    2008

  51. [60]

    Siddharth Singhal, Utkarsh Chawla, and Rajeev Shorey. 2020. Machine learning & concept drift based approach for malicious website detec- tion. In 2020 International Conference on COMmunication Systems & NETworkS (COMSNETS). IEEE, 582–585

    2020

  52. [61]

    Andrea Stocco, Michael Weiss, Marco Calzana, and Paolo Tonella

  53. [62]

    Sadayappan

    Kevin Stock, Louis-Noël Pouchet, and P. Sadayappan. 2012. Using Machine Learning to Improve Automatic Vectorization. ACM Trans. Archit. Code Optim. 8, 4, Article 50 (jan 2012), 23 pages. https://doi. org/10.1145/2086696.2086729

    2012

  54. [63]

    Wenlong Tang and Edward S Sazonov. 2014. Highly accurate recog- nition of human postures and activities through classification with rejection. IEEE journal of biomedical and health informatics (2014)

    2014

  55. [64]

    Yaniv Romano, Evan Patterson, and Emmanuel Candes. 2019. Confor- malized quantile regression. Advances in neural information processing systems 32 (2019)

    2019

  56. [65]

    Ben Taylor, Vicent Sanz Marco, Willy Wolff, Yehia Elkhatib, and Zheng Wang. 2018. Adaptive deep learning model selection on embedded systems. In Proceedings of the 19th ACM SIGPLAN/SIGBED International Conference on Languages, Compilers, and Tools for Embedded Systems . 31–43

    2018

  57. [66]

    Mauricio Sadinle, Jing Lei, and Larry Wasserman. 2019. Least ambigu- ous set-valued classifiers with bounded error levels. J. Amer. Statist. Assoc. 114, 525 (2019), 223–234

    2019

  58. [67]

    David Sculley, Gary Holt, Daniel Golovin, Eugene Davydov, Todd Phillips, Dietmar Ebner, Vinay Chaudhary, and Michael Young. 2014. Machine learning: The high interest credit card of technical debt. (2014)

    2014

  59. [68]

    Georgios Tournavitis, Zheng Wang, Björn Franke, and Michael FP O’Boyle. 2009. Towards a holistic approach to auto-parallelization: integrating profile-driven parallelism detection and machine-learning based mapping. ACM Sigplan notices 44, 6 (2009), 177–187

    2009

  60. [69]

    Foivos Tsimpourlas, Pavlos Petoumenos, Min Xu, Chris Cummins, Kim Hazelwood, Ajitha Rajan, and Hugh Leather. 2023. BenchDi- rect: A Directed Language Model for Compiler Benchmarks. In The 32nd International Conference on Parallel Architectures and Compilation Techniques (PACT)

    2023

  61. [70]

    Alexey Tsymbal. 2004. The problem of concept drift: definitions and related work. Computer Science Department, Trinity College Dublin 106, 2 (2004), 58

    2004

  62. [71]

    VenkataKeerthy, Rohit Aggarwal, Shalini Jain, Maunendra Sankar Desarkar, Ramakrishna Upadrasta, and Y

    S. VenkataKeerthy, Rohit Aggarwal, Shalini Jain, Maunendra Sankar Desarkar, Ramakrishna Upadrasta, and Y. N. Srikant. 2020. IR2VEC: LLVM IR Based Scalable Program Embeddings. ACM Trans. Archit. Code Optim. 17, 4, Article 32 (dec 2020), 27 pages. https://doi.org/10. 1145/3418463

    2020

  63. [72]

    Vladimir Vovk. 2012. Conditional validity of inductive conformal predictors. In Asian conference on machine learning . PMLR, 475–490

    2012

  64. [73]

    Vianney Taquet, Vincent Blot, Thomas Morzadec, Louis Lacombe, and Nicolas Brunel. 2022. MAPIE: an open-source library for distribution- free uncertainty quantification. arXiv preprint arXiv:2207.12274 (2022)

    2022 arXiv

  65. [74]

    Huanting Wang, Zhanyong Tang, Cheng Zhang, Jiaqi Zhao, Chris Cummins, Hugh Leather, and Zheng Wang. 2022. Automating re- inforcement learning architecture design for code optimization. In Proceedings of the 31st ACM SIGPLAN International Conference on Com- piler Construction. 129–143

    2022

  66. [75]

    Ronald A Thisted. 1998. What is a P-value. Departments of Statistics and Health Studies (1998)

    1998

  67. [76]

    Robert Tibshirani, Guenther Walther, and Trevor Hastie. 2001. Esti- mating the number of clusters in a data set via the gap statistic.Journal of the Royal Statistical Society: Series B (Statistical Methodology) 63, 2 (2001), 411–423

    2001

  68. [77]

    Zheng Wang and Michael FP O’Boyle. 2009. Mapping parallelism to multi-cores: a machine learning based approach. In Proceedings of the 14th ACM SIGPLAN symposium on Principles and practice of parallel programming. 75–84

    2009

  69. [78]

    Zheng Wang and Michael FP O’Boyle. 2010. Partitioning streaming parallelism for multi-cores: a machine learning based approach. In Proceedings of the 19th international conference on Parallel architectures and compilation techniques. 307–318

    2010

  70. [79]

    Zheng Wang and Michael O’Boyle. 2018. Machine learning in compiler optimization. Proc. IEEE 106, 11 (2018), 1879–1901

    2018

  71. [80]

    Petros Xanthopoulos, Panos M Pardalos, Theodore B Trafalis, Petros Xanthopoulos, Panos M Pardalos, and Theodore B Trafalis. 2013. Linear discriminant analysis. Robust data mining (2013), 27–33

    2013

  72. [81]

    Xueshuo Xie, Zongming Jin, Jiming Wang, Lei Yang, Ye Lu, and Tao Li

  73. [82]

    Huanting Wang, Zhanyong Tang, Shin Hwei Tan, Jie Wang, Yuzhe Liu, Hejun Fang, Chunwei Xia, and Zheng Wang. 2024. Combining Structured Static Code Information and Dynamic Symbolic Traces for Software Vulnerability Prediction. In Proceedings of the IEEE/ACM 46th International Co...

    2024

  74. [83]

    Shuangjiao Zhai, Zhanyong Tang, Petteri Nurmi, Dingyi Fang, Xi- aojiang Chen, and Zheng Wang. 2021. RISE: Robust Wireless Sens- ing Using Probabilistic and Statistical Assessments. In Proceedings of the 27th Annual International Conference on Mobile Computing and Networking (N...

    2021

  75. [84]

    H. Wang, G. Ye, Z. Tang, S. H. Tan, S. Huang, D. Fang, Y. Feng, L. Bian, and Z. Wang. 2021. Combining Graph-Based Learning With Automated Data Collection for Code Vulnerability Detection. IEEE Transactions on Information Forensics and Security 16 (2021), 1943–1958. https://doi...

    2021

  76. [85]

    Yiming Wang, Meng Hao, Hui He, Weizhe Zhang, Qiuyuan Tang, Xi- aoyang Sun, and Zheng Wang. 2024. DRLCap: Runtime GPU Frequency Capping with Deep Reinforcement Learning. IEEE Transactions on Sustainable Computing (2024)

    2024

  77. [91]

    Journal of Network and Computer Applications (2020)

    Confidence guided anomaly detection model for anti-concept drift in dynamic logs. Journal of Network and Computer Applications (2020)

    2020

  78. [92]

    Guixin Ye, Zhanyong Tang, Huanting Wang, Dingyi Fang, Jianbin Fang, Songfang Huang, and Zheng Wang. 2020. Deep Program Struc- ture Modeling Through Multi-Relational Graph-Based Learning. In Proceedings of the ACM International Conference on Parallel Architec- tures and Compila...

    2020

  79. [94]

    Yi Zhai, Yu Zhang, Shuo Liu, Xiaomeng Chu, Jie Peng, Jianmin Ji, and Yanyong Zhang. 2023. Tlp: A deep learning-based cost model for tensor program tuning. In Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating S...

    2023

  80. [95]

    Lianmin Zheng, Ruochen Liu, Junru Shao, Tianqi Chen, Joseph E Gonzalez, Ion Stoica, and Ameer Haj Ali. 2021. Tenset: A large-scale program performance dataset for learned tensor compilers. In Thirty- fifth Conference on Neural Information Processing Systems Datasets and Benchm...

    2021

  81. [2006]

    In Proceedings 20th IEEE International Parallel & Distributed Processing Symposium

    On the impact of data input sets on statistical compiler tuning. In Proceedings 20th IEEE International Parallel & Distributed Processing Symposium. IEEE, 8–pp

  82. [2014]

    In International Conference on Parallel Architectures and Compilation Techniques (PACT)

    OpenTuner: An Extensible Framework for Program Autotuning. In International Conference on Parallel Architectures and Compilation Techniques (PACT). Edmonton, Canada. http://groups.csail.mit.edu/ commit/papers/2014/ansel-pact14-opentuner.pdf

    2014

  83. [2017]

    In 26th International Conference on Parallel Architectures and Compilation Tech- niques (PACT)

    End-to-end deep learning of optimization heuristics. In 26th International Conference on Parallel Architectures and Compilation Tech- niques (PACT)

  84. [2018]

    arXiv preprint arXiv:1810.04805 (2018)

    Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)

    2018 arXiv

  85. [2020]

    In 2020 IEEE/ACM 42nd International Conference on Software Engineering (ICSE)

    Misbehaviour Prediction for Autonomous Driving Systems. In 2020 IEEE/ACM 42nd International Conference on Software Engineering (ICSE). 359–371

    2020

  86. [2021]

    arXiv preprint arXiv:2111.05672 (2021)

    Automatically detecting data drift in machine learning classifiers. arXiv preprint arXiv:2111.05672 (2021)

    2021 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.