Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

CodeImprove: Program Adaptation for Deep Code Models

T0 review · 3 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read CodeImprove claims that out-of-scope program inputs can be flagged by a layerwise sub-model validity score and rewritten by semantic-preserving transformations so that deep code models classify them correctly, with no retraining.

desk verdict Plausible inference-time adaptation idea with a solid evaluation skeleton, but the headline gains rest on a semantic-preservation assumption that one of the paper's own transformations violates, so the numbers are not yet interpretable. read the letter →

arxiv 2501.15804 v2 pith:YM5M5ZFJ submitted 2025-01-27 cs.SE

Ravishka Rathnasuriya , Zijie Zhao , Wei Yang This is my paper · ORCID
classification cs.SE
keywords inputvalidationadaptationprogramtransformationdeepcodemodelsuncertaintyestimationgeneticalgorithmsemantic-preservingout-of-scopedetection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a deployed deep code model can be made more accurate without retraining by adapting its inputs. It proposes a two-stage pipeline: first, a validity score built from layerwise sub-models flags programs the model is likely to misclassify; second, a genetic algorithm applies semantic-preserving program transformations to rewrite those programs into ones the model handles correctly. On vulnerability detection and defect prediction across three pre-trained code models, the pipeline reports accuracy gains up to 8.78% absolute and 51.28% relative, with input validation reaching an AUC of 0.924. A sympathetic reader would care because frequent retraining of code models is expensive in labeling and deployment, and input adaptation offers a cheaper alternative that leaves the original model untouched.

What carries the argument

The load-bearing object is the validity score from DSMG. Each sub-model $k$ keeps the original model's weights up to layer $k$ and adds a newly trained dense layer; for an input predicted as label $l_x$, the sub-model score is either $\mathrm{submodel}_k[l_x]$ plus the best-versus-second-best margin (if the prediction is correct) or $\mathrm{submodel}_k[l_x]$ minus the difference between the highest and the chosen probability (if the prediction is wrong). The final validity score is the weighted average of sub-model scores using Dissector-style weight growth. This score does two jobs: it is the oracle that decides which inputs need adaptation, and it is the fitness function that drives AES's genetic search, so the whole pipeline depends on it being better at identifying misclassifications than conventional confidence and uncertainty metrics.

What would settle it

Run a differential test on the full set of before-and-after program pairs that CodeImprove reports as corrected: compile both versions with the same compiler, execute them on the same inputs, and compare observable outputs and exit codes. The paper's own semantic-preservation check (RQ5) inspects only a single transformed example; if any pair whose prediction was "corrected" differs in observable behavior — for instance, a program whose printf debugging output was deleted under operator 15 — the label no longer applies to the transformed code, and the accuracy gain is an artifact rather than a genuine model improvement.

Watch

Extended reading notes

Core claim

The central claim is that out-of-scope program inputs — code snippets a trained model is prone to mishandle — can be reliably detected and then converted into in-scope inputs through syntax-level program transformation, all without modifying the model. Detection rests on a new validity score computed from Dropout-based Sub-Model Generation (DSMG): the original model is sliced at intermediate layers, each slice is capped with a newly trained dense classifier, and the agreement pattern of these sub-models' predictions (via a best-versus-second-best formula aggregated by weight growth) becomes the validity score. Adaptation rests on Adaptation by Evolutionary Search (AES), a genetic algorithm over fifteen listed transformations (renaming, loop rewrites, relational operator flips, and similar) guided by that validity score as fitness. The paper reports that this two-stage scheme improves three pre-trained models on two software-engineering tasks, correcting 23.1%–39.9% of mispredictions, and that the validity score distinguishes in-scope from out-of-scope inputs with AUC scores of 0.781–0.924, far above the best conventional uncertainty metric it tested (0.624).

Load-bearing premise

The load-bearing premise is that all fifteen transformations preserve program semantics for every test input, so ground-truth labels remain valid after rewriting; the paper asserts this without formal proof or differential testing, and one listed operator — deleting comments and debugging print statements — is not generally semantics-preserving.

Editorial extensions

If this is right

  • Deployed code models can be improved on new data without retraining, so agile projects can keep models effective as the codebase shifts.
  • The validity score gives a principled way to pick which inputs to send to a more expensive fallback, such as human review or a larger model.
  • The same pipeline transfers across models and tasks: it worked on CodeBERT, RoBERTa, and GraphCodeBERT for both vulnerability detection and defect prediction, so it is not tied to one architecture.
  • Because corrections come from input changes, the approach composes with any model and any future retraining schedule.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same machinery that repairs mispredictions could also be used adversarially: a search that finds validity-lowering transformations might generate semantic-preserving inputs that flip a model's prediction, so the paper's method doubles as a robustness probe.
  • The 0.924 AUC suggests sub-model disagreement encodes a general signal of model competence on code; a testable extension is to check whether the same score predicts performance on out-of-distribution projects or on code written by different developers.
  • The reported gains rest on an unproved semantics-preservation assumption; rerunning AES with differential testing as a constraint might reduce the correction rate but would make the numbers trustworthy.
  • The genetic search uses only three generations and a fixed operator list; larger budgets or learned transformation selection could push accuracy further if semantic equivalence is guaranteed.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes CodeImprove, a two-phase inference-time method for improving deep code models without retraining. The first phase, Dropout-based Sub-Model Generation (DSMG), builds layerwise sub-models from the original model and computes a validity score (Equations 1-3) to flag out-of-scope inputs. The second phase, Adaptation by Evolutionary Search (AES), uses a genetic algorithm over a list of 15 program transformations guided by the DSMG validity score to convert out-of-scope inputs into in-scope ones. The evaluation covers CodeBERT, RoBERTa, and GraphCodeBERT on two tasks, Devign vulnerability detection and CodeChef defect prediction, and reports accuracy improvements up to 8.78% (relative improvement 51.28%) and input-validation AUC up to 0.924. The paper claims all transformations are semantic-preserving.

Significance. If the semantic-preservation claim were rigorously established, this would be a useful and relatively novel contribution: an inference-time alternative to retraining for code models, together with a layerwise validity score that clearly outperforms standard uncertainty metrics (AUC 0.781-0.924 vs. at most 0.624 in Table I). The public artifact and reproducible setup are positive aspects. However, the central adaptation result depends on the transformed programs retaining the original ground-truth labels, and the current evidence for that is a single worked example. The significance is therefore conditional on a systematic semantic-preservation verification.

major comments (3)
  1. [Section IV-B, Table II, and Section VI-E (RQ5)] The semantic-preservation assumption is load-bearing and is not established. Operator 15, 'changeDeleteComments,' is described as 'Deleting statements that print debugging hints and comment.(e.g., printf())'; deleting a printf statement can change stdout, execution time, or side effects. For the CodeChef defect-prediction task, the ground-truth labels include 'wrong output,' 'timeout error,' and 'runtime error,' all of which are behavior-dependent, so a transformed program can legitimately have a different label than the original. RQ5 verifies only one example (Figure 3) and does not execute either program; Section VII itself lists 'lack of ground truths' as an internal threat. This is not a presentation issue: if transformations change program behavior, the reported accuracy gains are not attributable to adapting the model's input space but to changing the labels. The authors should either remove or restrict operator 15, run differential testing on the actual test inputs for all transformed programs, and report per-operator correction statistics.
  2. [Section V, threshold settings and result reporting] The threshold values c=0.3 for vulnerability detection and c=0.2 for defect prediction are fixed per task with no reported calibration procedure. These thresholds determine which inputs are adapted and when AES terminates, and they affect the reported CVR, MVR, CSR, and accuracy numbers. In addition, all reported results are single point estimates without error bars, confidence intervals, or repeated runs. Given that the genetic search is stochastic, the headline improvement of 8.78% should be accompanied by a sensitivity analysis over thresholds and seeds, or at least multiple independent runs with variance reported.
  3. [Section IV-A, Equations 1-3, and Section VII] The validity score in Equation 1/2 is defined relative to whether the original model's predicted label l_x is correct or incorrect, and Section VII states that 'mispredictions and correct predictions' are used to simulate out-of-scope and in-scope inputs. The paper does not state which data split is used to train the sub-model dense layers or to determine these correctness labels, nor whether the thresholds are chosen on the test set. If the test-set correctness labels or the test-set predictions participate in training or threshold selection, the reported AUC (0.781-0.924), CVR, and MVR are optimistically biased. Please clarify the data flow and use a held-out validation set for all calibration steps.
minor comments (5)
  1. [Table III vs. Table VI] For the CodeBERT vulnerability-detection configuration, Table III reports CSR/MCR as 39.9/2.6 while Table VI reports 39.9/4.5 for the same CodeImprove setting; these numbers should be reconciled.
  2. [Table VI] The entry for CodeImprove-rand with RoBERTa on vulnerability detection reads '6.11.1/1.5' and appears to be a formatting typo for '6.1/1.5'.
  3. [Throughout] There are typos and grammar issues that should be corrected: 'genetics algorithms' in the abstract, 'covert' in Section IV-B, 'upto' in the abstract, a duplicate 'the' in Section IV-A, and 'hiss guidance' in the acknowledgments.
  4. [Equations 1-2] The notation for B(submodel_k) is confusing because the two cases in Equation 2 are not mutually exclusive in the way the surrounding text presents them; please restate the definition of B more clearly and define ls and lh before using them.
  5. [Figure 3] The RQ5 example shows several transformations (declaration splitting, loop rewriting, relational/incremental rewriting) but it does not show execution output or behavior of the original vs. transformed code; a systematic differential test would be much stronger.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation found: the reported accuracy and AUC gains are measured against held-out ground-truth labels, and the validity score is a learned surrogate rather than a renamed version of the target.

full rationale

CodeImprove's central claims are empirically anchored rather than definitionally forced. The input-validation AUC (0.924) is computed by labeling positive and negative samples from the correspondence between model predictions and ground-truth labels (Section III-A), and the final accuracy and RI gains in Table III are measured on the held-out test set against the original ground-truth labels after transformation. The DSMG validity score is a learned surrogate: sub-models reuse frozen early layers of the original model and add newly trained dense layers, and the score is then used as the genetic-algorithm fitness. This is surrogate optimization, not a circular reduction, because the reported outcome is the held-out label accuracy, not the fitness value itself. The thresholds (0.3 and 0.2) are selected on validation data, and AUC is threshold-independent, so the reported detection AUC is not an artifact of that fit. I also checked the self-citations: reference [8] is co-authored by one of the present authors, but it is cited only as related work on uncertainty under distribution shift and is not the load-bearing justification for CodeImprove's validity score or transformation operators. Section VII does flag a real limitation: 'the lack of ground truths for distinguishing out-of-scope inputs from in-scope inputs' and the reliance on 15 asserted semantic-preserving transformation rules. That is an empirical validity threat, not circularity: the transformations are not derived from the ground-truth labels, and the semantic-preservation claim is checked, though thinly, in RQ5. If operator 15 deletes printf statements, a transformed program may have different behavior, which would make some accuracy gains spurious; that is a correctness risk to be addressed by differential testing, not a circular derivation.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central estimate rests on hand-set thresholds, unspecified sub-model hyperparameters, and unverified semantic-preservation of transformations. No new physical entities are introduced.

free parameters (5)
  • validation threshold c = 0.3 (vulnerability detection), 0.2 (defect prediction)
    Set based on the validation score without a reported calibration protocol; determines which inputs are adapted and therefore drives all accuracy gains.
  • maximum AES iterations = 3
    Fixed by the authors; a larger search budget could change CSR and MCR and the reported improvements.
  • crossover rates = 0.16, 0.33, 0.66, 1.0
    Listed as configuration choices, but no results by crossover rate are reported, so their effect on the headline numbers is unclear.
  • number of transformation operators N = 15 (default)
    The sensitivity study shows CSR and RI vary with N, so the default choice affects the headline results.
  • sub-model layer choices and dropout rate = not stated
    DSMG requires selecting intermediate layers k and dropout parameters; these are not specified and affect the validity score.
assumptions (4)
  • domain assumption In-scope inputs can be identified with correct predictions and out-of-scope inputs with mispredictions of the original model.
    Used to label positives and negatives for AUC and to define the validation problem; stated in Section VII as a rough estimation.
  • ad hoc to paper The 15 transformations preserve program semantics for all test inputs.
    No proof or differential testing is provided; RQ5 verifies a single example, and Table II operator 15 deletes printf and comment statements.
  • domain assumption Agreement among layerwise sub-model predictions indicates original model correctness.
    Central to the DSMG validity score; the paper motivates it with observations but does not prove it.
  • domain assumption A frozen first-k-layers encoder plus a newly trained dense layer is a valid surrogate of the original model's handling capability at layer k.
    Sub-model generation relies on this without a formal guarantee or comparison to other surrogate designs.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CodeImprove: Program Adaptation for Deep Code Models." pith.science (2026). https://pith.science/paper/YM5M5ZFJ

@misc{pith2026250115804,
  author       = {Pith},
  title        = {Pith review of: CodeImprove: Program Adaptation for Deep Code Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YM5M5ZFJ}},
  note         = {Machine review of arXiv:2501.15804}
}
read the original abstract

Leveraging deep learning (DL)-based code analysis tools to solve software engineering tasks is becoming increasingly popular. Code models often suffer performance degradation due to various reasons (e.g., code data shifts). Retraining is often required to address these issues, but frequent model updates are costly in labeling and deployment. In this paper, we explore an alternative solution: Adapting the program inputs to the code models. This can be achieved by two steps: 1) input validation that focuses on identifying whether an input is an out-of-scope input program that are beyond a model's handling capability, and 2) input adaptation that adapts out-of-scope inputs to become in-scope inputs. Validating program input is challenging, as current techniques focus on continuous inputs such as image data and fail with discrete inputs like code data, which have unique characteristics and are processed differently by deep learning models. Adapting out-of-scope programs is also challenging due to their vast search spaces. Therefore, in this paper, we propose CodeImprove, which distinguishes out-of-scope from normal inputs and converts such out-of-scope inputs back to in-scope inputs through program transformation. In particular, we propose a validity score metric to identify out-of-scope inputs and leverage genetic algorithms to apply semantic preserving program transformation to convert out-of-scope inputs to in-scope inputs. Our experimental results show CodeImprove can enhance up to 8.78% of accuracy, and 51.28% of relative improvements in three code models on two SE tasks. Additionally, our input validation is promising in detecting out-of-scope inputs (AUC score of 0.924).

Figures

Figures reproduced from arXiv: 2501.15804 by the authors.

Figure 1
Figure 1. Overview of CodeImprove IV. DESIGN OF CODEIMPROVE [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Overview of Sub-Model Generation Sub-model Validity Measurement. Equation 1 outlines the computation of the validity score, which relies on understand￾ing the processing of inputs across layers. For a given input X fed into a DL model M with n labels, M predicts X as label lx. Let submodelk be the softmax probability vector associated with the sub-model k. To calculate the validity score, we differentiate between tw… view at source ↗
Figure 3
Figure 3. Additional examples are on our project website due to space restrictions [44] [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Time Overhead of CodeImprove RQ6 - What is the overhead of CodeImprove in adapting a program to DL models? CodeImprove was highly efficient in adapting an out-of￾scope input through semantic preserving program transfor￾mations in real-time (1.2TPS - 2.04TPS). VII. THRE…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Framework for On the Fly Input Refinement for Deep Learning Models

    cs.SE 2025-02 unverdicted novelty 4.0 of 10

    A proposal to reduce deep learning mispredictions at inference by validating, transforming, and searching over inputs, with preliminary evidence only for code classification.

Reference graph

Works this paper leans on

58 extracted references · 20 canonical work pages · cited by 1 Pith paper

  1. [1]

    On-the-fly improving perfor- mance of deep code models via input denoising,

    Z. Tian, J. Chen, and X. Zhang, “On-the-fly improving perfor- mance of deep code models via input denoising,” arXiv preprint arXiv:2308.09969, 2023

    arXiv 2023

  2. [2]

    Natural attack for pre-trained models of code,

    Z. Yang, J. Shi, J. He, and D. Lo, “Natural attack for pre-trained models of code,” in Proceedings of the 44th International Conference on Software Engineering, ser. ICSE ’22. New York, NY , USA: Association for Computing Machinery, 2022, p. 1482–1493. [Online]. Available: https://doi.org/10.1145/3510003.3510146

    arXiv 2022

  3. [3]

    Challenging Machine Learning-based Clone Detectors via Semantic-preserving Code Transformations,

    W. Zhang, S. Guo, H. Zhang, Y . Sui, Y . Xue, and Y . Xu, “Challenging Machine Learning-based Clone Detectors via Semantic-preserving Code Transformations,” IEEE Transactions on Software Engineering, vol. 49, no. 5, pp. 3052–3070, May 2023

    2023

  4. [4]

    Codexglue: A machine learning benchmark dataset for code understanding and generation,

    S. Lu, D. Guo, S. Ren, J. Huang, A. Svyatkovskiy, A. Blanco, C. Clement, D. Drain, D. Jiang, D. Tang et al., “Codexglue: A machine learning benchmark dataset for code understanding and generation,” arXiv preprint arXiv:2102.04664, 2021

    arXiv 2021

  5. [5]

    Robustness, security, privacy, explainability, efficiency, and usability of large language models for code,

    Z. Yang, Z. Sun, T. Z. Yue, P. Devanbu, and D. Lo, “Robustness, security, privacy, explainability, efficiency, and usability of large language models for code,” arXiv preprint arXiv:2403.07506, 2024

    arXiv 2024

  6. [6]

    Adversarial examples for models of code,

    N. Yefet, U. Alon, and E. Yahav, “Adversarial examples for models of code,” Proceedings of the ACM on Programming Languages, vol. 4, no. OOPSLA, pp. 1–30, 2020

    work page 2020

  7. [7]

    Codes: towards code model generalization under distribution shift,

    Q. Hu, Y . Guo, X. Xie, M. Cordy, M. Papadakis, L. Ma, and Y . Le Traon, “Codes: towards code model generalization under distribution shift,” in International Conference on Software Engineering (ICSE): New Ideas and Emerging Results (NIER), 2023

    work page 2023

  8. [8]

    Estimating predictive uncertainty under program data distribution shift,

    Y . Li, S. Chen, and W. Yang, “Estimating predictive uncertainty under program data distribution shift,” arXiv preprint arXiv:2107.10989, 2021

    arXiv 2021

Show all 58 references
  1. [9]

    Tailoring programs for static analysis via program transformation,

    R. van Tonder and C. L. Goues, “Tailoring programs for static analysis via program transformation,” in Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, 2020, pp. 824–834

    2020

  2. [10]

    T-fuzz: fuzzing by program transformation,

    H. Peng, Y . Shoshitaishvili, and M. Payer, “T-fuzz: fuzzing by program transformation,” in 2018 IEEE Symposium on Security and Privacy (SP). IEEE, 2018, pp. 697–710

    2018

  3. [11]

    BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” May 2019

    2019

  4. [12]

    DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter,

    V . Sanh, L. Debut, J. Chaumond, and T. Wolf, “DistilBERT, a distilled version of BERT: Smaller, faster, cheaper and lighter,” Feb. 2020

    2020

  5. [13]

    Graphcodebert: Pre- training code representations with data flow,

    D. Guo, S. Ren, S. Lu, Z. Feng, D. Tang, S. Liu, L. Zhou, N. Duan, A. Svyatkovskiy, S. Fu, M. Tufano, S. K. Deng, C. Clement, D. Drain, N. Sundaresan, J. Yin, D. Jiang, and M. Zhou, “Graphcodebert: Pre- training code representations with data flow,” 2021

    2021

  6. [14]

    Evaluating Large Language Models Trained on Code,

    M. Chen, J. Tworek, H. Jun, Q. Yuan, Pinto et al., “Evaluating Large Language Models Trained on Code,” Jul. 2021

    2021

  7. [15]

    Data Augmentation by Program Trans- formation,

    S. Yu, T. Wang, and J. Wang, “Data Augmentation by Program Trans- formation,” Journal of Systems and Software, vol. 190, p. 111304, Aug. 2022

    2022

  8. [16]

    Self-checking deep neural networks in deployment,

    Y . Xiao, I. Beschastnikh, D. S. Rosenblum, C. Sun, S. Elbaum, Y . Lin, and J. S. Dong, “Self-checking deep neural networks in deployment,” 2021

    2021

  9. [17]

    Repairing failure-inducing inputs with input reflection,

    Y . Xiao, Y . Lin, I. Beschastnikh, C. Sun, D. S. Rosenblum, and J. S. Dong, “Repairing failure-inducing inputs with input reflection,” in The 37th IEEE/ACM International Conference on Automated Software Engineering (ASE). IEEE, 2022

    2022

  10. [18]

    On calibration of modern neural networks,

    C. Guo, G. Pleiss, Y . Sun, and K. Q. Weinberger, “On calibration of modern neural networks,” 2017

    2017

  11. [19]

    Dissector: Input val- idation for deep learning applications by crossing-layer dissection,

    H. Wang, J. Xu, C. Xu, X. Ma, and J. Lu, “Dissector: Input val- idation for deep learning applications by crossing-layer dissection,” in Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, 2020, pp. 727–738

    2020

  12. [20]

    A baseline for detecting misclassified and out-of-distribution examples in neural networks,

    D. Hendrycks and K. Gimpel, “A baseline for detecting misclassified and out-of-distribution examples in neural networks,” 2018

    2018

  13. [21]

    Dropout as a bayesian approximation: Representing model uncertainty in deep learning,

    Y . Gal and Z. Ghahramani, “Dropout as a bayesian approximation: Representing model uncertainty in deep learning,” in international conference on machine learning. PMLR, 2016, pp. 1050–1059

    2016

  14. [22]

    code2vec: Learn- ing distributed representations of code,

    U. Alon, M. Zilberstein, O. Levy, and E. Yahav, “code2vec: Learn- ing distributed representations of code,” Proceedings of the ACM on Programming Languages, vol. 3, no. POPL, pp. 1–29, 2019

    2019

  15. [23]

    Quantifying uncertainties in natural language processing tasks,

    Y . Xiao and W. Y . Wang, “Quantifying uncertainties in natural language processing tasks,” in Proceedings of the AAAI conference on artificial intelligence, vol. 33, no. 01, 2019, pp. 7322–7329

    2019

  16. [24]

    Towards better confidence estimation for neural models,

    V . T. Vasudevan, A. Sethy, and A. R. Ghias, “Towards better confidence estimation for neural models,” in ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019, pp. 7335–7339

    2019

  17. [25]

    Addressing failure prediction by learning model confidence,

    C. Corbi `ere, N. Thome, A. Bar-Hen, M. Cord, and P. P´erez, “Addressing failure prediction by learning model confidence,” Advances in Neural Information Processing Systems, vol. 32, 2019

    2019

  18. [26]

    R. M. Monarch, Human-in-the-Loop Machine Learning: Active learning and annotation for human-centered AI. Simon and Schuster, 2021

    2021

  19. [27]

    Unsupervised risk estimation using only conditional independence structure,

    J. Steinhardt and P. S. Liang, “Unsupervised risk estimation using only conditional independence structure,” Advances in Neural Information Processing Systems, vol. 29, 2016

    2016

  20. [28]

    A mathematical theory of communication,

    C. E. Shannon, “A mathematical theory of communication,” The Bell system technical journal, vol. 27, no. 3, pp. 379–423, 1948

    1948

  21. [29]

    Codeimprove repository

    CodeImprove, “Codeimprove repository.” [Online]. Available: https: //github.com/CodeImprove/CodeImprove

  22. [30]

    Towards evaluating the robustness of neural networks,

    N. Carlini and D. Wagner, “Towards evaluating the robustness of neural networks,” in 2017 ieee symposium on security and privacy (sp). Ieee, 2017, pp. 39–57

    2017

  23. [31]

    Distillation as a defense to adversarial perturbations against deep neural networks,

    N. Papernot, P. McDaniel, X. Wu, S. Jha, and A. Swami, “Distillation as a defense to adversarial perturbations against deep neural networks,” in 2016 IEEE symposium on security and privacy (SP). IEEE, 2016, pp. 582–597

    2016

  24. [32]

    Intriguing properties of neural networks,

    C. Szegedy, W. Zaremba, I. Sutskever, J. Bruna, D. Erhan, I. Goodfellow, and R. Fergus, “Intriguing properties of neural networks,” arXiv preprint arXiv:1312.6199, 2013

    2013 arXiv

  25. [33]

    On the effectiveness of defensive distillation,

    N. Papernot and P. McDaniel, “On the effectiveness of defensive distillation,” arXiv preprint arXiv:1607.05113, 2016

    2016 arXiv

  26. [34]

    Simple and scalable predictive uncertainty estimation using deep ensembles,

    B. Lakshminarayanan, A. Pritzel, and C. Blundell, “Simple and scalable predictive uncertainty estimation using deep ensembles,” Advances in neural information processing systems, vol. 30, 2017

    2017

  27. [35]

    Random search algorithms,

    Z. B. Zabinsky et al., “Random search algorithms,” Department of Industrial and Systems Engineering, University of Washington, USA, 2009

    2009

  28. [36]

    Hill-climbing search,

    B. Selman and C. P. Gomes, “Hill-climbing search,” Encyclopedia of cognitive science, vol. 81, p. 82, 2006

    2006

  29. [37]

    Search-based software engineering,

    M. Harman and B. F. Jones, “Search-based software engineering,” Information and software Technology, vol. 43, no. 14, pp. 833–839, 2001

    2001

  30. [38]

    Bfgs optimization for faster and automated supervised learning,

    t. I. N. N. C. C. S. International Neural Network Society (INNS), R. Battiti, and F. Masulli, “Bfgs optimization for faster and automated supervised learning,” in International Neural Network Conference: July 9–13, 1990 Palais Des Congres—Paris—France. Springer, 1990, pp. 757–760

    1990

  31. [39]

    The relationship between precision-recall and roc curves,

    J. Davis and M. Goadrich, “The relationship between precision-recall and roc curves,” in Proceedings of the 23rd international conference on Machine learning, 2006, pp. 233–240

    2006

  32. [40]

    A survey of uncertainty in deep neural networks,

    J. Gawlikowski, C. R. N. Tassi, M. Ali, J. Lee, M. Humt, J. Feng, A. Kruspe, R. Triebel, P. Jung, R. Roscheret al., “A survey of uncertainty in deep neural networks,” Artificial Intelligence Review, vol. 56, no. Suppl 1, pp. 1513–1589, 2023

    2023

  33. [41]

    Unveiling code pre-trained models: Investigating syntax and semantics capacities,

    W. Ma, S. Liu, M. Zhao, X. Xie, W. Wang, Q. Hu, J. Zhang, and Y . Liu, “Unveiling code pre-trained models: Investigating syntax and semantics capacities,” ACM Transactions on Software Engineering and Methodology, 2024

    2024

  34. [42]

    Densely connected convolutional networks,

    G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 4700–4708

    2017

  35. [43]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 770–778

    2016

  36. [44]

    Codeimprove

    CodeImprove, “Codeimprove.” [Online]. Available: https://codeimprove. github.io/CodeImprove-io/

  37. [45]

    Genetic algorithm: Review and application,

    M. Kumar, D. M. Husain, N. Upreti, and D. Gupta, “Genetic algorithm: Review and application,” Available at SSRN 3529843, 2010

    2010

  38. [46]

    Genetic algorithms,

    S. Forrest, “Genetic algorithms,” ACM computing surveys (CSUR), vol. 28, no. 1, pp. 77–80, 1996

    1996

  39. [47]

    A genetic algorithm tutorial,

    D. Whitley, “A genetic algorithm tutorial,” Statistics and computing, vol. 4, pp. 65–85, 1994

    1994

  40. [48]

    Devign: Effective vul- nerability identification by learning comprehensive program semantics via graph neural networks,

    Y . Zhou, S. Liu, J. Siow, X. Du, and Y . Liu, “Devign: Effective vul- nerability identification by learning comprehensive program semantics via graph neural networks,” Advances in neural information processing systems, vol. 32, 2019. 12

    2019

  41. [49]

    Convolutional neural networks on assembly code for predicting software defects,

    A. V . Phan and M. Le Nguyen, “Convolutional neural networks on assembly code for predicting software defects,” in 2017 21st Asia Pacific Symposium on Intelligent and Evolutionary Systems (IES), 2017, pp. 37–42

    2017

  42. [50]

    CodeBERT: A Pre-Trained Model for Programming and Natural Languages,

    Z. Feng, D. Guo, D. Tang, N. Duan, X. Feng, M. Gong, L. Shou, B. Qin, T. Liu, D. Jiang, and M. Zhou, “CodeBERT: A Pre-Trained Model for Programming and Natural Languages,” Sep. 2020

    2020

  43. [51]

    RoBERTa: A Robustly Optimized BERT Pretraining Approach,

    Y . Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V . Stoyanov, “RoBERTa: A Robustly Optimized BERT Pretraining Approach,” Jul. 2019

    2019

  44. [52]

    Code difference guided adversarial example generation for deep code models,

    Z. Tian, J. Chen, and Z. Jin, “Code difference guided adversarial example generation for deep code models,” 2023

    2023

  45. [53]

    Natural attack for pre-trained models of code,

    Z. Yang, J. Shi, J. He, and D. Lo, “Natural attack for pre-trained models of code,” in Proceedings of the 44th International Conference on Software Engineering, 2022, pp. 1482–1493

    2022

  46. [54]

    Towards robustness of deep program processing models—detection, estimation, and enhancement,

    H. Zhang, Z. Fu, G. Li, L. Ma, Z. Zhao, H. Yang, Y . Sun, Y . Liu, and Z. Jin, “Towards robustness of deep program processing models—detection, estimation, and enhancement,” ACM Trans. Softw. Eng. Methodol., vol. 31, no. 3, apr 2022. [Online]. Available: https://doi.org/10.114...

    2022 doi

  47. [55]

    A twofold siamese network for real-time object tracking,

    A. He, C. Luo, X. Tian, and W. Zeng, “A twofold siamese network for real-time object tracking,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 4834–4843

    2018

  48. [56]

    Variable kernel density estimation,

    G. R. Terrell and D. W. Scott, “Variable kernel density estimation,” The Annals of Statistics, pp. 1236–1265, 1992

    1992

  49. [57]

    Generating adversarial examples for holding robustness of source code processing models,

    H. Zhang, Z. Li, G. Li, L. Ma, Y . Liu, and Z. Jin, “Generating adversarial examples for holding robustness of source code processing models,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 34, no. 01, 2020, pp. 1169–1176

    2020

  50. [58]

    Generating adversarial computer programs using optimized obfuscations,

    S. Srikant, S. Liu, T. Mitrovska, S. Chang, Q. Fan, G. Zhang, and U.-M. O’Reilly, “Generating adversarial computer programs using optimized obfuscations,” arXiv preprint arXiv:2103.11882, 2021. 13

    2021 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.