Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

Enhancing LLM-Based Code Generation with Complexity Metrics: A Feedback-Driven Approach

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper claims that the static complexity of generated code predicts whether it passes its tests, and that prompting a failed model to shift the most predictive complexity metrics measurably raises Pass@1, most for GPT-3.5-turbo.

desk verdict A plausible but not yet proven idea: complexity-metric feedback may help LLM code repair, but the paper never shows the complexity signal is the active ingredient. read the letter →

arxiv 2505.23953 v1 pith:WBWPR55M submitted 2025-05-29 cs.SE cs.AI

Melika Sepidband , Hamed Taherkhani , Song Wang , Hadi Hemmati This is my paper
classification cs.SEcs.AI
keywords codegenerationcomplexitymetricsPass@1iterativefeedbackShapleyvalueslogisticregressionLLMrepairHumanEval
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that the complexity of LLM-generated code is not a byproduct but a usable signal: the value of standard complexity metrics on a first attempt predicts whether that attempt will pass its tests, and telling the model to change those values on a failed attempt makes the next attempt pass more often. The authors measure 53 static complexity metrics — cyclomatic complexity, Halstead volume and effort, line counts, and related quantities — on generated solutions, train a logistic regression to predict pass/fail, and use Shapley values (a per-metric credit-assignment ranking) to pick the five most predictive metrics per dataset. Failed solutions are sent back to the model with the instruction to regenerate with different values for those metrics, for up to five rounds, with internal evaluation run on test cases generated by a strong model so that the ground-truth tests never leak into the loop. On HumanEval the feedback lifts GPT-3.5-turbo's Pass@1 from 0.56 to 0.76, a relative gain of 35.71% against 12.5% for an execution-feedback-only baseline, and layering the same feedback on top of an existing agent workflow yields further gains on the harder BigCodeBench benchmark. A sympathetic reader cares because the method is cheap and model-agnostic, and it works best exactly where models are weakest.

What carries the argument

The load-bearing mechanism is the complexity-feedback loop. It computes 53 static complexity metrics (McCabe's cyclomatic complexity, the number of linearly independent control-flow paths; Halstead length, volume, difficulty, effort, and time, which are size-and-effort measures built from counts of operators and operands; the maintainability index; line counts; counts of loops, comparisons, literals, variables, math operations, and nested-block depth; and per-keyword frequencies) on each generated solution. A logistic regression on the pass/fail label ranks the metrics, and Shapley values select the five most influential ones per dataset. For any failed solution, the model is prompted, "Please ensure that your generated code has different values for the following complexity metrics," and this is repeated for at most five iterations, with pass/fail judged on test cases generated by a second model to avoid leaking the dataset's own tests into the loop. The entire argument rests on both steps working: the metrics must actually be predictive of correctness, and the prompt 'different values' must actually steer regeneration toward the complexity profile of correct code.

What would settle it

A control experiment settles it: under identical iteration counts and internal test cases, prompt the model with a matched generic instruction — 'your code is incorrect, rewrite it differently,' or 'make the solution simpler or more elaborate' — with no complexity metrics named. If Pass@1 gains match the complexity-feedback condition, the metrics are not the causal channel. A complementary check requires no new experiments: recompute the five target metrics on every regenerated sample; if their values do not shift relative to the failed sample while Pass@1 still improves, the stated pathway is not operating.

Watch

Extended reading notes

Core claim

In the paper's own framing, the central discovery is that the five complexity metrics most predictive of code correctness — identified per dataset by logistic regression on 53 metrics and ranked by Shapley values — can be turned into an effective repair prompt. The prediction step is real: a logistic regression over complexity metrics classifies Pass@1 with accuracy up to 0.921 (GPT-4o on HumanEval), and the metric distributions differ systematically between successes and failures, with GPT-4o and Llama 3.1 tending to fail with over-complex solutions while GPT-3.5-turbo tends to fail with overly simple ones. The repair step then consists of prompting the failed model to regenerate code whose values for those five metrics are different, iterating up to five times; across HumanEval, MBPP, and LeetCode this consistently outperforms both zero-shot generation and iterative regeneration with execution feedback alone, and combining it with an agent-based method raises BigCodeBench Pass@1 from 0.30 to 0.36 for GPT-4o and from 0.39 to 0.48 for GPT-o3 mini. The paper's claim is that complexity-aware feedback is a distinct and effective channel for interactive code repair — not a replacement for execution feedback, but a complement that helps most when a model's failures are attributable to getting a task's complexity profile wrong.

Load-bearing premise

The load-bearing premise is that instructing the model to produce different values for the selected complexity metrics is what improves the code — that the metric shift, rather than the generic pressure to alter a failed answer, causes the observed Pass@1 gains, and that the regenerated code actually moves those metrics in the intended direction.

Editorial extensions

If this is right

  • Pass@1 can be predicted from static complexity metrics alone with high accuracy on complexity-sensitive benchmarks (up to 0.921 for GPT-4o on HumanEval), giving a cheap, execution-free early warning of likely failure.
  • Complexity-directed regeneration beats naive iterative regeneration on every dataset and model tested, so the metric channel adds signal beyond 'the tests failed'.
  • The benefit concentrates on weaker models: GPT-3.5-turbo's HumanEval Pass@1 rose 35.71% versus 12.5% for the execution-feedback baseline, suggesting complexity feedback helps most where initial solutions are structurally off-target.
  • Layering the feedback on top of the Reflexion agent lifts BigCodeBench Pass@1 from 0.30 to 0.36 for GPT-4o and from 0.39 to 0.48 for GPT-o3 mini over five iterations, so the mechanism composes with existing repair agents.
  • The method's value is dataset-dependent — HumanEval and LeetCode are complexity-sensitive while MBPP is largely not — so complexity feedback is a targeted tool rather than a universal fix.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Untested by the paper, but decisive for its interpretation: there is no ablation with a matched generic instruction such as 'your code is incorrect, rewrite it differently,' so the observed gains could in principle come from generic pressure to alter a failed answer rather than from the complexity metrics themselves.
  • The Shapley-ranked metrics implicitly define a per-dataset complexity profile of correct code; a natural next step, not taken here, is to prompt toward a target metric value rather than merely toward a different value, which would turn the loop into a guided search.
  • The RQ2 asymmetry — GPT-4o and Llama 3.1 fail with over-complex code while GPT-3.5-turbo fails with over-simple code — suggests a directional prompt ('simplify' for some models, 'elaborate' for others) would likely outperform the one-size-fits-all 'different values' instruction the paper uses.
  • A cheaper deployment would use the trained complexity classifier as a pre-filter to decide which generated candidates are worth running tests on, saving execution budget in agent pipelines.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper investigates whether standard code complexity metrics of LLM-generated code can predict Pass@1 and whether feeding these metrics back to the LLM improves code generation. The authors train logistic regression models with feature selection (including SHAP) on complexity metrics from generated code, report predictive accuracies, analyze metric distributions between successful and failed outputs, and propose an iterative feedback loop in which failed code is regenerated with a prompt to change the five most important SHAP-selected metrics. They evaluate on HumanEval, MBPP, LeetCode, and BigCodeBench with GPT-4o, GPT-3.5-turbo, Llama 3.1, and GPT-o3 mini, including an integration with Reflexion, and report Pass@1 improvements over zero-shot and iterative regeneration baselines.

Significance. If the causal claim held, this would be a low-cost, model-agnostic way to improve code generation by guiding LLMs away from complexity profiles associated with failure. The paper has clear strengths: a released artifact, multiple benchmarks and models, and a transparent pipeline. However, the causal interpretation is currently under-supported because the feedback condition is not separated from generic regeneration, and the RQ1 predictive evidence is not benchmarked against a majority-class baseline. The work is therefore promising but not yet established.

major comments (4)
  1. [Section III.E.2 and Algorithm 1 (Steps 8-9)] The feedback prompt asks the LLM to produce 'different values' for the five SHAP-selected metrics, with no direction and no verification that the regenerated code's metric values actually change in the intended direction. The only baseline for RQ3 is 'naive regeneration' (white rows in Table III), which does not control for the generic instruction to alter the previous output. Consequently, the observed Pass@1 gains (e.g., GPT-3.5-turbo HumanEval, 0.56 to 0.76 vs. 0.56 to 0.63 in baseline) could arise from generic pressure to change the code rather than from the complexity information. Add a control condition with a generic 'modify your previous solution' prompt, and report measured metric changes on a sample of regenerated codes.
  2. [Section IV.A, Table II] Logistic regression accuracies are reported without a majority-class baseline, balanced accuracy, or confidence intervals. Several values are close to or below the base rate (e.g., GPT-4o MBPP 0.74 vs. a 0.70 base rate from Table III; GPT-3.5-turbo MBPP 0.623 vs. 0.67 base rate). The claim of a 'clear correlation' (Answer to RQ1) is therefore overstated. Report majority-class accuracy, AUC, or odds ratios with uncertainty.
  3. [Section IV.C, Tables III and IV] Iteration 0 (zero-shot) values differ between the two experimental rows for the same model/dataset, e.g., GPT-4o MBPP 0.68 vs. 0.70 and GPT-4o LeetCode 0.91 vs. 0.92. Since iteration 0 is the same initial generation, this inconsistency indicates unreported stochasticity or a different evaluation setup, and it undermines the paired comparison. All reported numbers appear to come from a single run or an unreported aggregation; error bars or repeated trials are needed to assess whether differences are significant.
  4. [Section III.E.3 and Algorithm 1] The internal evaluation uses test cases generated by GPT-4o, but the paper does not assess the fidelity of these generated tests relative to the original test suite. If the generated tests are lenient, the algorithm may stop iterating on code that would fail the original tests, biasing the final Pass@1 in an unmeasured way. Report agreement rates between LLM-generated and original tests, or justify why this does not affect the comparison with the baseline.
minor comments (5)
  1. [Section IV.A] 'Shaply' is a typo for 'Shapley' (two occurrences in the answer to RQ1).
  2. [Tables III and IV] Model naming is inconsistent between 'GPT-o3 mini' and 'gpt-o3-mini'; standardize across text and tables.
  3. [Figure 4] The figure has nine subplots with small fonts; consider making the key metrics or per-dataset patterns more readable, perhaps with a separate plot per dataset.
  4. [Section III.A] The paper states the LeetCode dataset has 2,360 problems but only 561 were used; the selection criterion (problems with verified correct code) should be stated explicitly in the main text or with a reference to the repository.
  5. [Algorithm 1, line 14] The variable 'M' is used without prior definition; replace it with 'metrics' for clarity.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the RQ3 improvements are measured out-of-sample by execution on original test cases, and the paper's self-citations are present but not load-bearing.

full rationale

The paper's claimed derivation chain does not contain a definitional or fitted-value circularity. The RQ1 logistic-regression/Shapley analysis is trained on complexity metrics computed from generated code with execution-based pass@1 labels under 5-fold cross-validation, and RQ3 selects the five most important metrics on the training folds only and applies them to held-out evaluation folds (Algorithm 1, lines 8-23), with final Pass@1 measured by execution against the original dataset test cases (Step 10). The reported gains (e.g., GPT-3.5-turbo on HumanEval from 0.56 to 0.76) are therefore empirical out-of-sample results rather than predictions forced by the fitted model, so the improvement claim does not reduce to the regression input. The 53-metric set is standard, externally defined, and self-described in Table I, so the citations to the authors' own prior work ([29], [30], which include co-author Song Wang) are not load-bearing; the same applies to related-work citations [24], [26], and [27] that share authors with this paper. The method's genuine weakness is construct validity rather than circularity: the feedback prompt only asks for 'different values' for the five metrics without specifying direction (Section III.E.2), the paper never verifies that the regenerated code's metric values actually moved in the intended direction, and the naive-regeneration baseline does not control for generic code-alteration pressure. These are missing-ablation concerns that belong under correctness risk, not under circularity, and no step in the claimed derivation is equivalent to its own input by construction.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the transferability of SHAP-selected metrics across folds and on the assumption that LLMs can alter specific code characteristics in response to metric names. The paper provides no sensitivity analysis for the number of iterations or metrics, and no verification that the complexity feedback actually changes the metric values as intended.

free parameters (3)
  • Maximum iterations N = 5
    Chosen by hand as the stopping criterion for the iterative feedback loop (Algorithm 1); the paper provides no sensitivity analysis for this value.
  • Number of feedback metrics = 5
    The five most influential complexity metrics per dataset, selected by SHAP; the paper does not test whether 5 is optimal or whether the ranking is stable across folds.
  • Per-dataset SHAP-selected metric sets = Varies per dataset, e.g., Halstead length, vocabulary, effort, LOC, math operations for HumanEval with GPT-4o
    These are fitted on the training fold and then used as feedback on the evaluation fold; they are data-dependent outputs of the logistic regression, not fixed constants.
assumptions (4)
  • domain assumption Complexity metrics of generated code are predictive of code correctness across LLMs and datasets.
    This is the premise RQ1 tests, but the paper assumes a stable relationship when using the same metrics for feedback in RQ3.
  • domain assumption SHAP values from a logistic regression trained on a training fold identify the metrics most likely to be causally linked to correctness in a held-out fold.
    The method relies on SHAP rankings transferring across folds; the paper does not validate stability.
  • domain assumption LLM-generated test cases (via GPT-4o) are a valid internal oracle during iteration.
    Stated in Section III-F3 and acknowledged as a threat in Section V; if these tests are systematically easier or misaligned, the iterative selection may be biased.
  • domain assumption LLMs can alter specific complexity metrics in response to natural language prompts.
    The prompt 'ensure that your generated code has different values for the following complexity metrics' assumes the model can meaningfully change those characteristics; no evidence is provided that the metrics of regenerated code actually change as requested.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Enhancing LLM-Based Code Generation with Complexity Metrics: A Feedback-Driven Approach." pith.science (2026). https://pith.science/paper/WBWPR55M

@misc{pith2026250523953,
  author       = {Pith},
  title        = {Pith review of: Enhancing LLM-Based Code Generation with Complexity Metrics: A Feedback-Driven Approach},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WBWPR55M}},
  note         = {Machine review of arXiv:2505.23953}
}
read the original abstract

Automatic code generation has gained significant momentum with the advent of Large Language Models (LLMs) such as GPT-4. Although many studies focus on improving the effectiveness of LLMs for code generation, very limited work tries to understand the generated code's characteristics and leverage that to improve failed cases. In this paper, as the most straightforward characteristic of code, we investigate the relationship between code complexity and the success of LLM generated code. Using a large set of standard complexity metrics, we first conduct an empirical analysis to explore their correlation with LLM's performance on code generation (i.e., Pass@1). Using logistic regression models, we identify which complexity metrics are most predictive of code correctness. Building on these findings, we propose an iterative feedback method, where LLMs are prompted to generate correct code based on complexity metrics from previous failed outputs. We validate our approach across multiple benchmarks (i.e., HumanEval, MBPP, LeetCode, and BigCodeBench) and various LLMs (i.e., GPT-4o, GPT-3.5 Turbo, Llama 3.1, and GPT-o3 mini), comparing the results with two baseline methods: (a) zero-shot generation, and (b) iterative execution-based feedback without our code complexity insights. Experiment results show that our approach makes notable improvements, particularly with a smaller LLM (GPT3.5 Turbo), where, e.g., Pass@1 increased by 35.71% compared to the baseline's improvement of 12.5% on the HumanEval dataset. The study expands experiments to BigCodeBench and integrates the method with the Reflexion code generation agent, leading to Pass@1 improvements of 20% (GPT-4o) and 23.07% (GPT-o3 mini). The results highlight that complexity-aware feedback enhances both direct LLM prompting and agent-based workflows.

Figures

Figures reproduced from arXiv: 2505.23953 by the authors.

Figure 1
Figure 1. Comparison of incorrect code generated by the LLM (left), the correct code generated by our approach (middle), and [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Overview of Complexity-Aware Feedback for Enhanced LLM Code Generation [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. The distribution of Halstead Length by target value [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Bar plot showing the median differences in complexity metrics between target values (pass@1 = 1 and pass@1 = 0) [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Shapley values illustrating the importance of various [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Throttling Web Agents Using Reasoning Gates

    cs.AI 2025-09 conditional novelty 6.0 of 10

    Rebus-based reasoning gates, puzzles built from random word/domain clue sets, impose token costs on LM web agents that are up to 9.2x the generator's cost.

Reference graph

Works this paper leans on

34 extracted references · 17 canonical work pages · cited by 1 Pith paper

  1. [1]

    No man is an island: Towards fully automatic programming by code search, code generation and program repair,

    Q. Zhang, C. Fang, Y . Shang, T. Zhang, S. Yu, and Z. Chen, “No man is an island: Towards fully automatic programming by code search, code generation and program repair,”arXiv preprint arXiv:2409.03267, 2024

    arXiv 2024

  2. [2]

    Gpt-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkatet al., “Gpt-4 technical report,”arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  3. [3]

    Chain-of-thought prompting elicits reasoning in large language models,

    J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V . Le, D. Zhouet al., “Chain-of-thought prompting elicits reasoning in large language models,”Advances in neural information processing systems, vol. 35, pp. 24 824–24 837, 2022

    2022

  4. [4]

    Reflex- ion: Language agents with verbal reinforcement learning,

    N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao, “Reflex- ion: Language agents with verbal reinforcement learning,”Advances in Neural Information Processing Systems, vol. 36, pp. 8634–8652, 2023

    2023

  5. [5]

    Cyclomatic complexity,

    C. Ebert, J. Cain, G. Antoniol, S. Counsell, and P. Laplante, “Cyclomatic complexity,”IEEE software, vol. 33, no. 6, pp. 27–29, 2016

    work page 2016

  6. [6]

    Software complexity analysis using halstead metrics,

    T. Hariprasad, G. Vidhyagaran, K. Seenu, and C. Thirumalai, “Software complexity analysis using halstead metrics,” in2017 international con- ference on trends in electronics and informatics (ICEI). IEEE, 2017, pp. 1109–1113

    work page 2017

  7. [7]

    OpenAI, “GPT-4o,” https://openai.com/index/hello-gpt-4o/, 2024, ac- cessed: 2024-05-13

    work page 2024

  8. [8]

    Evaluating large language models trained on code,

    M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. D. O. Pinto, J. Kaplan, H. Edwards, Y . Burda, N. Joseph, G. Brockmanet al., “Evaluating large language models trained on code,”arXiv preprint arXiv:2107.03374, 2021

    arXiv 2021

Show all 34 references
  1. [9]

    Logistic regression,

    M. P. LaValley, “Logistic regression,”Circulation, vol. 117, no. 18, pp. 2395–2399, 2008

    2008

  2. [10]

    Gpt-3.5-turbo,

    OpenAI, “Gpt-3.5-turbo,” https://platform.openai.com/docs/models/ gpt-3.5-turbo, 2021, 2021-09

    2021

  3. [11]

    The llama 3 herd of models,

    A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Yang, A. Fanet al., “The llama 3 herd of models,”arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  4. [12]

    Program synthesis with large language models,

    J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. Cai, M. Terry, Q. Leet al., “Program synthesis with large language models,”arXiv preprint arXiv:2108.07732, 2021

    2021 arXiv

  5. [13]

    Leetcode dataset,

    Greengerong, “Leetcode dataset,” https://huggingface.co/datasets/ greengerong/leetcode, 2023

    2023

  6. [14]

    Introduction to the shapley value,

    A. E. Roth, “Introduction to the shapley value,”The Shapley value, vol. 1, 1988

    1988

  7. [15]

    Bigcodebench: Benchmarking code generation with diverse function calls and complex instructions,

    T. Y . Zhuo, M. C. Vu, J. Chim, H. Hu, W. Yu, R. Widyasari, I. N. B. Yusuf, H. Zhan, J. He, I. Paulet al., “Bigcodebench: Benchmarking code generation with diverse function calls and complex instructions,” arXiv preprint arXiv:2406.15877, 2024

    2024 arXiv

  8. [16]

    Code llama: Open foundation models for code,

    B. Roziere, J. Gehring, F. Gloeckle, S. Sootla, I. Gat, X. E. Tan, Y . Adi, J. Liu, R. Sauvestre, T. Remezet al., “Code llama: Open foundation models for code,”arXiv preprint arXiv:2308.12950, 2023

    2023 arXiv

  9. [17]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y . Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosaleet al., “Llama 2: Open foundation and fine-tuned chat models,”arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  10. [18]

    Gpt-o3-mini,

    OpenAI, “Gpt-o3-mini,” https://openai.com/index/openai-o3-mini/, 2025, 2025-01-31

    2025

  11. [19]

    Parsel: Algorithmic reasoning with language models by composing decompositions,

    E. Zelikman, Q. Huang, G. Poesia, N. Goodman, and N. Haber, “Parsel: Algorithmic reasoning with language models by composing decompositions,”Advances in Neural Information Processing Systems, vol. 36, pp. 31 466–31 523, 2023

    2023

  12. [20]

    Anpl: towards natural programming with interactive decomposition,

    D. Huang, Z. Nan, X. Hu, P. Jin, S. Peng, Y . Wen, R. Zhang, Z. Du, Q. Guo, Y . Puet al., “Anpl: towards natural programming with interactive decomposition,”Advances in Neural Information Processing Systems, vol. 36, pp. 69 404–69 440, 2023

    2023

  13. [21]

    Dynamic llm-agent network: An llm-agent collaboration framework with agent team op- timization,

    Z. Liu, Y . Zhang, P. Li, Y . Liu, and D. Yang, “Dynamic llm-agent network: An llm-agent collaboration framework with agent team op- timization,”arXiv preprint arXiv:2310.02170, 2023

    2023 arXiv

  14. [22]

    Agentcoder: Multi-agent-based code generation with iterative testing and optimisation,

    D. Huang, J. M. Zhang, M. Luck, Q. Bu, Y . Qing, and H. Cui, “Agentcoder: Multi-agent-based code generation with iterative testing and optimisation,”arXiv preprint arXiv:2312.13010, 2023

    2023 arXiv

  15. [23]

    Ldb: A large language model debugger via verifying runtime execution step-by-step,

    L. Zhong, Z. Wang, and J. Shang, “Ldb: A large language model debugger via verifying runtime execution step-by-step,”arXiv preprint arXiv:2402.16906, 2024

    2024 arXiv

  16. [24]

    Epic: Cost-effective search-based prompt engineering of llms for code generation,

    H. Taherkhani, M. Sepidband, H. V . Pham, S. Wang, and H. Hemmati, “Epic: Cost-effective search-based prompt engineering of llms for code generation,”arXiv preprint arXiv:2408.11198, 2024

    2024 arXiv

  17. [25]

    A survey on metric of software complexity,

    S. Yu and S. Zhou, “A survey on metric of software complexity,” in 2010 2nd IEEE International conference on information management and engineering. IEEE, 2010, pp. 352–356

    2010

  18. [26]

    A pragmatic approach for hyper-parameter tuning in search-based test case generation,

    S. Zamani and H. Hemmati, “A pragmatic approach for hyper-parameter tuning in search-based test case generation,”Empirical Software Engi- neering, vol. 26, pp. 1–35, 2021

    2021

  19. [27]

    An empirical study on bug severity estimation using source code metrics and static analysis,

    E. Mashhadi, S. Chowdhury, S. Modaberi, H. Hemmati, and G. Uddin, “An empirical study on bug severity estimation using source code metrics and static analysis,”Journal of Systems and Software, vol. 217, p. 112179, 2024

    2024

  20. [28]

    Analysis and modeling conditional mutual dependency of metrics in software defect prediction using latent variables,

    N. S. Harzevili and S. H. Alizadeh, “Analysis and modeling conditional mutual dependency of metrics in software defect prediction using latent variables,”Neurocomputing, vol. 460, pp. 309–330, 2021

    2021

  21. [29]

    Automatically learning semantic features for defect prediction,

    S. Wang, T. Liu, and L. Tan, “Automatically learning semantic features for defect prediction,” inProceedings of the 38th international confer- ence on software engineering, 2016, pp. 297–308

    2016

  22. [30]

    Continuous software bug prediction,

    S. Wang, J. Wang, J. Nam, and N. Nagappan, “Continuous software bug prediction,” inProceedings of the 15th ACM/IEEE International Sym- posium on Empirical Software Engineering and Measurement (ESEM), 2021, pp. 1–12

    2021

  23. [31]

    Feature selection, l 1 vs. l 2 regularization, and rotational invariance,

    A. Y . Ng, “Feature selection, l 1 vs. l 2 regularization, and rotational invariance,” inProceedings of the twenty-first international conference on Machine learning, 2004, p. 78

    2004

  24. [32]

    Gene selection for cancer classification using support vector machines,

    I. Guyon, J. Weston, S. Barnhill, and V . Vapnik, “Gene selection for cancer classification using support vector machines,”Machine learning, vol. 46, pp. 389–422, 2002

    2002

  25. [33]

    Correlation-based feature selection for machine learning,

    M. A. Hall, “Correlation-based feature selection for machine learning,” Ph.D. dissertation, The University of Waikato, 1999

    1999

  26. [34]

    A unified approach to interpreting model predictions,

    S. Lundberg, “A unified approach to interpreting model predictions,” arXiv preprint arXiv:1705.07874, 2017

    2017 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.