{"id":"05836055-a7df-4664-aa20-16cf6f99f7be","arxiv_id":"2505.15088","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"GPT-4 detected command injection in popular Python projects with 75.5% accuracy and F1 74.5%, missing 15 real cases and beating Bandit on precision.","lead":"The paper tested four large language models on 190 Python functions from six major open-source projects to see whether the models can find command injection vulnerabilities and write tests that actually prove the attack. It found GPT-4 the most accurate detector at about 75% accuracy, and gives developers a direct cost and accuracy comparison for deciding whether LLM-based scanning can replace or complement traditional static tools.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported metrics assume an independent ground truth, but 90 'safe' verdicts are never test-executed, false positives rest on a single failed test authored by the same model, and no reachability check proves attacker control; all headline numbers depend on this oracle.","rationale":"The reader's weakest assumption identifies the same load-bearing concern: the ground-truth oracle is under-specified, partially self-referential, and lacks a reachability check. My inspection of the full text and appendix confirms this is the linchpin of the paper. The arithmetic internally checks out, and the executed tests for many 'vulnerable' cases are real evidence at function level, so the concern is not that the numbers are fabricated. It is that the numbers describe agreement with a label set whose construction is circular or invisible: the same model that produces the prediction writes the test used to confirm it; 'safe' cases are never tested despite Section 5.1 claiming they were; and 'vulnerable' is defined without any call-chain or data-flow evidence that an attacker controls the sink. Every headline metric and the RQ5 comparison with Bandit are conditional on this oracle. Because the reader already reached CONDITIONAL on essentially these grounds, my analysis does not change the verdict; it sharpens the condition to require an independent, blind, reachability-aware re-labeling of all 190 cases plus release of the dataset.","tokens_in":20374,"tokens_out":10205,"duration_ms":81151,"concrete_test":"Perform an independent audit of all 190 cases: release the promised dataset (code, prompts, generated tests, labels) and have at least two security researchers, blind to GPT-4's answers, independently re-derive the 'Actually vulnerable?' labels using a documented protocol. The protocol should execute or review a test for every case GPT-4 or a candidate tool flags as vulnerable, run an independent taint/static analysis (e.g., a CodeQL command-injection query) on all cases labeled safe, and require a reachability path from a public caller to the dangerous sink before labeling a function vulnerable. Report inter-rater agreement (Cohen's kappa), then recompute TP/FP/TN/FN and the metrics in Tables 3, 4, and 10 from the consensus labels. If the recomputed confusion matrix differs materially from the published one, the RQ1 and RQ5 claims do not survive.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The Answer to RQ1 (accuracy 75.5%, precision 68.4%, recall 81.7%, F1 74.5%) is only as sound as the 'Actually vulnerable?' labels in the appendix. That column is not an independent oracle. For the 100 functions GPT-4 judged vulnerable, the confirming evidence is a security test GPT-4 itself generated and the authors manually repaired (Section 4.4); a single failed injection attempt is then treated as proof the function is not vulnerable (31 FPs). For the 90 functions GPT-4 judged safe, no security test is executed at all: Section 4.3 states that if GPT-4 determines a function is safe, no vulnerability is detected, and no test is generated. Section 5.1 nevertheless claims the 75 true negatives were 'confirmed by the absence of vulnerabilities in security tests', contradicting the method. The 15 false negatives rest on a manual review whose protocol, reviewer count, and blindness to the model's answers are never reported. In addition, none of the 'vulnerable' labels include a reachability check showing an attacker-controlled value actually reaches the dangerous sink from a project entry point; the motivating example is only shown to be injectable when called directly with a malicious pid. If any of these label sources shift, the confusion matrix in Table 3, the metrics in Table 4, and the Bandit comparison in Table 10 all change.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents an empirical study in which four LLMs (GPT-4, GPT-4o, Claude 3.5 Sonnet, DeepSeek-R1) are prompted to analyze 190 Python functions, drawn from six popular open-source projects, that contain dangerous methods from the Semgrep command-injection list. GPT-4's 100 positive verdicts are followed up by LLM-generated security tests that the authors manually repaired and executed; the 90 negative verdicts receive no test execution. The authors report a confusion matrix (67 TP, 31 FP, 75 TN, 15 FN, 2 invalid), derive accuracy 75.5%, precision 68.4%, recall 81.7%, and F1 74.5%, compare the four models, and compare GPT-4 against Bandit, claiming that the LLM approach reduces false positives and false negatives.","tokens_in":20594,"tokens_out":2968,"duration_ms":27162,"significance":"If the reported ground truth were independently established, the study would be a useful empirical data point: it provides a 190-function command-injection candidate set from six widely used projects, actually executes injection attempts for the 100 functions GPT-4 labels as vulnerable, compares four LLMs on the same candidate set, and supplies a Bandit baseline. The executable-testing branch for the positive cases is a genuine strength, and the analysis of list-argument subprocess calls as a source of false negatives (Section 5.2, Listing 3) is a concrete, credible finding. However, the central metrics are only as sound as the ground-truth labels, and the paper's own method section shows that those labels are not an independent oracle for the negative cases.","major_comments":[{"comment":"The ground truth for the 90 functions GPT-4 judged safe is not empirically validated. Section 4.3 states that if GPT-4 determines a function is safe, no security test is generated, yet Section 5.1 reports 75 true negatives as \"confirmed by the absence of vulnerabilities in security tests.\" This is a direct contradiction. The 75 TN and 15 FN labels rest on a manual review whose protocol, reviewer count, and blindness to the model's answers are never reported. Because TN and FN enter accuracy, precision, recall, and F1, all headline numbers and the subsequent Bandit comparison depend on an unspecified oracle. The authors must either execute adversarial tests for the negative cases or describe an independent, reproducible ground-truth labeling procedure.","section":"Sections 4.3 and 5.1, Tables 3 and 4"},{"comment":"Even for the 100 positive-verdict cases, the confirming evidence is not independent of the system under test: the security tests are generated by the same LLM, then manually repaired by the authors, and a single failed injection attempt is treated as proof that the function is not vulnerable. Test failures can result from harness errors, missing dependencies, an incorrect attack payload, or an unreachable sink rather than from absence of a vulnerability. Additionally, no reachability check is reported showing that an attacker-controlled value actually reaches the dangerous sink from a project entry point; the motivating example in Section 2.1 only demonstrates injection when the function is called directly with a malicious pid. The paper should add an explicit reachability/taint analysis or an independent manual audit for every label in the appendix.","section":"Sections 4.4, 5.1, and Appendix ground-truth column"},{"comment":"The claim that the LLM approach outperforms Bandit by reducing false positive and false negative rates is load-bearing but inherits the unresolved ground-truth problem. Bandit's 103 false positives are judged against the same labels whose negative cases were never test-executed and whose positive cases were validated by repaired LLM-generated tests. A fair comparison requires the same independent oracle for both tools; as written, a shift in the appendix labels would change Table 3, Table 4, and Table 10 together.","section":"Section 5.5, Table 10"},{"comment":"The false-negative analysis is internally inconsistent in its case count: the text first says there are 15 false negatives, then says it analyzed \"these 14 cases,\" while Table 5 lists 15 rows (1 Scikit-learn + 1 Django + 13 PyTorch). The description of 10 subprocess-related cases plus 5 eval/exec cases also sums to 15. This inconsistency must be corrected, and the analysis should cover all 15 cases or explicitly exclude one with justification.","section":"Section 5.2, Table 5"}],"minor_comments":[{"comment":"The generated test file name appears as \"test file.txt\" rather than a proper Python identifier such as \"test_file.txt\"; please fix the typo and ensure the figure caption gives the full workflow.","section":"Section 2.1.2, Figure 1b"},{"comment":"The contributions claim the dataset is available on GitHub, but no repository URL is provided anywhere in the manuscript; please add a link or state the repository name for reproducibility.","section":"Section 1, dataset availability"},{"comment":"The security-test-generation comparison reports only counts of directly runnable tests; reporting the total number of tests that were executable after modification and the repair effort would make the comparison more informative.","section":"Section 5.4.2, Figure 6"},{"comment":"The internal-threats paragraph addresses un-runnable tests but does not mention the circularity of using LLM-generated tests as ground truth for the same LLM's verdicts; this should be acknowledged and mitigated.","section":"Section 7, threats to validity"},{"comment":"The tables use abbreviated column headers such as \"No. of cases\" and \"GPT's answer\" inconsistently across the appendix; please unify the header style and add a legend explaining the \"Invalid\" and \"Pending\" entries.","section":"Tables 7-10"}],"recommendation":"major_revision","confidential_remarks":"The paper is honest in reporting the validation loop, which helps the reviewer trust the authors' intentions, but the central quantitative claims cannot be accepted until the ground-truth oracle is made independent and reproducible. The main fix is within the manuscript's scope: either execute tests for the negative cases or provide a detailed, blinded manual audit with reachability analysis, and re-derive all tables from that oracle. I do not see evidence of misconduct; the problem is methodological completeness."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a readable empirical measurement with one genuinely new observation, but its headline numbers rest on a ground-truth oracle that is half self-referential and half undocumented. Take the 75.5% accuracy and the Bandit comparison as conditional until the 'No' branch is validated.\n\nWhat's actually new: the 190-function candidate set from six popular projects, per-model metrics for GPT-4, GPT-4o, Claude 3.5 Sonnet and DeepSeek-R1, cost and test-runnability figures, and the observation that GPT-4 routinely misses list-typed args passed to subprocess.run/Popen. That last point is the most useful thing in the paper.\n\nWhat it does well: the executed injection tests for the 100 'Yes' cases are real evidence—they run actual payloads and check for file deletion or command execution. The arithmetic checks out; the confusion matrices sum and the metrics recompute. The motivating example (get_child_pids) is well chosen and clearly explained.\n\nThe soft spots are concentrated in the validation loop, and the stress-test note lands. Section 4.3 states that if GPT-4 says 'No', no security test is generated, yet Section 5.1 claims the 75 true negatives were 'confirmed by the absence of vulnerabilities in security tests.' That is a direct contradiction. The 15 false negatives rest on a manual review with no protocol, reviewer count, or blindness to the model's answers. None of the 'vulnerable' labels include a reachability check—the motivating example is only shown injectable when called directly. The threats-to-validity section covers only invalid tests and model/project coverage, not the label-construction loop. The dataset is promised in the abstract but not accessible. There are also smaller inconsistencies: RQ2 says 14 cases when the table lists 15; the Answer to RQ5 says the method reduces false negatives, but Table 10 shows GPT-4 has 15 FN vs Bandit's 1. Temperature is reported for GPT models only.\n\nThese are fixable in revision, but they mean the headline numbers should not be taken at face value. The core idea—that LLM-generated executable tests provide a check on LLM vulnerability verdicts—is sound and worth evaluating, but the 'No' branch needs the same treatment as the 'Yes' branch, or at least a documented independent review.\n\nWho it's for: security teams comparing LLM-based scanning to Bandit, and SE researchers working on LLM evaluation methodology. It deserves peer review—the dataset and the list-args finding are worth refereeing—but it needs major revision before acceptance.","headline":"A useful but under-validated empirical baseline: the GPT-4 vs Bandit numbers rest on a ground-truth oracle that is half self-referential, so treat the headline metrics as conditional.","tokens_in":21209,"tokens_out":4370,"would_cite":false,"duration_ms":31607,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that GPT-4, prompted to analyze isolated Python functions and generate executable security tests, detects command injection vulnerabilities with 75.5% accuracy and beats Bandit.","keywords":["command injection","large language models","GPT-4","vulnerability detection","Python","security testing","static analysis"],"falsifier":"Have independent security researchers, blind to GPT-4's answers, manually audit all 190 candidate functions and check, for each dangerous call, whether an attacker actually controls the argument and whether the function is reachable in the real project. If the audit overturns even a modest share of the 75 'true negatives' or the 67 'true positives,' the reported accuracy, precision, recall, F1, and the Bandit comparison would shift.","tokens_in":20075,"feed_emoji":"🐍","tokens_out":4793,"duration_ms":35293,"temperature":0.7,"pith_summary":"This paper asks whether a large language model can do the work of a security scanner for a specific bug class: command injection in Python. The authors take 190 functions from six widely used open-source projects, ask GPT-4 to judge each one vulnerable or safe, and then try to confirm each 'vulnerable' judgement with an executable security test that GPT-4 itself writes. They report that GPT-4 reaches 75.5% accuracy, 68.4% precision, 81.7% recall, and an F1 score of 74.5% on this candidate set, and that this beats the traditional static-analysis tool Bandit on accuracy, precision, and F1. If this holds, LLM-based analysis is a credible complement to static scanners for finding command injection flaws in Python code.","feed_headline":"GPT-4 detects 75.5% of Python command injection cases","feed_subtitle":"On 190 functions from six popular open-source projects, it beats the static tool Bandit on precision and F1.","key_machinery":"The load-bearing mechanism is a pipeline that turns an LLM into a testable security oracle. First, a script selects functions containing any of 26 dangerous methods from the Semgrep catalog, such as eval(), exec(), subprocess.run(), and os.system(); this yields the 190 candidates. For each candidate, GPT-4 is asked, with a 'mimic-in-the-background' prompt, whether the function is vulnerable; if it says yes, the same model writes a unittest-based security test that attempts a real injection, and the authors manually repair tests that need environment fixes before running them. The executed test result then serves as the ground-truth label used to score the model.","core_discovery":"The central claim is that GPT-4, prompted to analyze isolated Python functions and to generate executable security tests for any function it flags, can detect command injection vulnerabilities in real open-source code with an accuracy of 75.5%, a precision of 68.4%, a recall of 81.7%, and an F1 score of 74.5%. On the same 190 candidate functions, Bandit reports 81 true positives and 103 false positives, giving it lower accuracy, precision, and F1; GPT-4 also finds two vulnerabilities Bandit misses. The paper further claims that among four LLMs, GPT-4 is the best detector while DeepSeek-R1 generates the most directly executable security tests.","pith_inferences":["A direct consequence the authors leave implicit is that pairing a high-precision detector (GPT-4) with a high-executability test writer (DeepSeek-R1) could outperform either model alone; this hybrid is testable on the released 190-function dataset.","Because the dataset is built from six projects' snapshots, the 190 functions are a frozen benchmark; running the same prompts on later versions of those projects would test whether the 75.5% accuracy is stable as code evolves.","The reported false-negative pattern suggests that a rule-based pre-filter for dangerous methods combined with an LLM judge could be improved by adding explicit checks for list-form subprocess arguments and attacker-influenced global variables."],"forward_implications":["Developers can run the LLM pipeline alongside Bandit to cut false positives in command-injection triage, at a reported cost of about $14 and 76 minutes for 190 functions.","LLM-generated security tests can serve as a first-pass confirmation signal, although the paper reports only 55 of GPT-4's 100 tests run without manual repairs.","The list-type subprocess call pattern, e.g., subprocess.run(args) with a list of strings, is a known blind spot that a detector should treat as a red flag, not dismiss.","Teams choosing an LLM for this task get different best models for detection versus test generation: GPT-4 for judging, DeepSeek-R1 for writing runnable tests."],"supporting_citations":[{"why":"Supplies the traditional static-analysis baseline the LLM is compared against.","marker":"[8]"},{"why":"Provides the catalog of 26 dangerous Python methods used to select candidate functions.","marker":"[41]"},{"why":"Introduces the 'mimic-in-the-background' prompting method adopted for GPT-4 prompts.","marker":"[42]"},{"why":"Defines the GPT-4 model being evaluated.","marker":"[14]"},{"why":"One of the four compared LLMs; best test-generation performer.","marker":"[17]"}],"fun_headline_variants":["GPT-4 spots 75.5% of Python command injection flaws","GPT-4 beats Bandit on Python injection detection","LLM analysis finds 75.5% of Python command injection cases","GPT-4 finds 75.5% of injection vulns in top Python projects"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire accuracy, precision, and F1 story depends on whether the 'actually vulnerable?' labels are right, and those labels come from a process the paper does not independently audit: tests written by GPT-4 itself for flagged functions, and an unreported manual review for the functions GPT-4 called safe.","fun_headline_variants_meta":{"raw":{"variants":["GPT-4 spots 75.5% of Python command injection flaws","GPT-4 beats Bandit on Python injection detection","LLM analysis finds 75.5% of Python command injection cases","GPT-4 finds 75.5% of injection vulns in top Python projects"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000589,"raw_usage":{"total_tokens":2753,"prompt_tokens":919,"completion_tokens":1834,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":535,"completion_tokens_details":{"reasoning_tokens":1756}},"tokens_in":535,"tokens_out":1834,"duration_ms":12026,"temperature":1.0,"reasoning_tokens":1756,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T15:25:13.965352+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Have independent security researchers, blind to GPT-4's answers, manually audit all 190 candidate functions and check, for each dangerous call, whether an attacker actually controls the argument and whether the function is reachable in the real project. If the audit overturns even a modest share of the 75 'true negatives' or the 67 'true positives,' the reported accuracy, precision, recall, F1, and the Bandit comparison would shift.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the traditional static-analysis baseline the LLM is compared against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the catalog of 26 dangerous Python methods used to select candidate functions."}],"review_version":1}