{"id":"72311ed7-c5fd-4ae1-a56c-021876a25273","arxiv_id":"2512.04259","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Using real ChatGPT conversation logs, code generated by the model is frequently insecure and users rarely request security-related code.","lead":"This study analyzed over 82,000 real ChatGPT conversations from the WildChat dataset and found that AI-generated code is frequently insecure—vulnerable regexes, unsafe memory functions, SQL injection, and fake package imports appear regularly. It also found that users rarely ask ChatGPT for secure code, with implications for how coding tools should protect users.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"User-intent finding rests on an unvalidated zero-shot classifier; if its Secure Coding category has low recall, the paper's distinctive claim that users rarely ask for secure code is unsupported.","rationale":"The reader's identified weakest assumption is correct and is indeed the most load-bearing concern. The paper's main security findings (ReDoS rate, unsafe memory functions, hallucinated imports) are supported by multiple established tools and are consistent with prior synthetic studies, so even if some rates shift slightly, the central security claim likely stands. However, the distinctive user-intent finding—that users rarely ask for secure code—is presented prominently in the abstract and conclusion, and it relies solely on an unvalidated zero-shot classifier. No accuracy, no validation set, no human agreement, and an extremely small Secure Coding category make this finding fragile. The proposed concrete test would directly measure whether the classifier can reliably detect security-oriented requests. If it cannot, the paper should be revised to present the user-intent result as preliminary or remove it from the headline claims. Since the reader's verdict is already CONDITIONAL, our assessment does not change the verdict, but it reinforces the need for the condition.","tokens_in":13957,"tokens_out":5253,"duration_ms":49264,"concrete_test":"Take a stratified random sample of 500 initial and 500 follow-up user messages from WildCode_EN (excluding code). Have two annotators independently label them using the paper's category definitions (provided in the GitHub repo). Measure inter-annotator agreement (e.g., Cohen's kappa), then compare the bart-large-mnli predictions to the consensus labels. Report per-category precision/recall, especially for 'Secure Coding'. If the classifier's recall for Secure Coding is below 0.7, or if humans find at least twice as many security-related queries as the classifier, the user-intent conclusions in Secs. 5.2–5.4 should be retracted or heavily hedged.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's novel and abstract-level claim that 'users don't ask many questions about the security characteristics' (Sec. 5.2) rests entirely on a zero-shot intent classifier (bart-large-mnli) that is never validated. The authors provide no validation set, accuracy, precision/recall, or human agreement for this classifier. Categories were hand-defined and fed as candidate labels; the tie-aware rounding to two decimals is ad hoc. The 'Secure Coding' category receives only 21 initial labels out of tens of thousands of coding conversations (Table 10) and only 6 follow-up labels in buggy-code conversations (Sec. 5.4). If the NLI model fails to recognize paraphrases like 'make this safe' or 'is this vulnerable to attacks?' as Secure Coding, the headline observation that users rarely request secure code—a key distinct contribution of this paper—could be a measurement artifact. Given that the security findings are consistent with prior synthetic studies and the user-intent result is wholly dependent on this unvalidated classifier, this is the most load-bearing weak point.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper reports an empirical study of real ChatGPT-generated code extracted from the WildChat dataset. It constructs a code corpus (82,843 conversations containing code), performs syntax checking on six languages, applies OpenGrep security rules for hash functions, SQL injection, RNG, deserialization, and memory safety, runs four ReDoS detectors on extracted regexes, checks Python/JavaScript imports for package hallucinations, and finally classifies English user queries into intent categories using zero-shot NLI. Headline numbers include 29.4% ReDoS-vulnerable regexes, 14.85% of C/C++ snippets using unsafe memory functions, 20.61% of hash-function conversations triggering weakness rules, and about 14.4% of Python imports hallucinated. The paper concludes that LLM-generated code is frequently insecure and that users rarely request security-focused code.","tokens_in":14167,"tokens_out":6863,"duration_ms":60026,"significance":"If the measurements are sound, this is a valuable complement to earlier synthetic-prompt studies: it uses authentic user–ChatGPT interactions, externalizes vulnerability detection to third-party scanners, and provides a public dataset for reuse. The honest treatment of known over- and undercounting in the security analysis is a strength. However, the distinctive user-intent contribution is currently supported only by an unvalidated zero-shot classifier, and the ReDoS aggregation table is internally inconsistent as printed. The paper is therefore significant in scope but not yet reliable in its strongest claims.","major_comments":[{"comment":"The conclusion that users rarely ask about security rests entirely on zero-shot classification with bart-large-mnli using hand-defined categories. No validation set, accuracy, precision/recall, human agreement, or error analysis is reported. The 'Secure Coding' category receives only 21 initial labels (Table 10) and 6 follow-up labels (Section 5.4). If the NLI model has low recall for paraphrases such as 'make this safe' or 'is this vulnerable to attacks?', the headline observation in the abstract and Section 5.2 could be a measurement artifact. This needs to be supported by a manually annotated sample with inter-annotator agreement, or the claim should be explicitly downgraded to a classifier-based observation.","section":"§5.2, Table 10"},{"comment":"The paper states it evaluates code 'with respect to correctness', but the correctness analysis in Section 4.2 is only a syntax check using py_compile, eslint, javac, gcc, php -l, and Roslyn. Syntactic validity is not functional or semantic correctness; a snippet can parse and still be wrong. Please replace the correctness terminology with 'syntactic validity' throughout, or provide an additional functional correctness evaluation (e.g., testing a sample of snippets against assertions).","section":"§4.2 and abstract"},{"comment":"The aggregation of ReDoS results is not reproducible from Table 9. For C/CPP, SafeRegex alone reports 40 vulnerable regexes out of 80, but the Total Vuln. column reports 36; for Java, SafeRegex reports 160 out of 226, while Total Vuln. reports 47; for Python, SafeRegex reports 309 out of 753, while Total Vuln. reports 212. If Total Vuln. is a union of the tools, it cannot be smaller than the per-tool counts; if it is an intersection, a manually confirmed subset, or something else, that definition and the supporting counts must be given. As printed, the 29.4% headline rate is not defensible from the table.","section":"§4.4, Table 9"},{"comment":"The Discussion says the detection 'likely overcounts some issues' while also 'undercounting others', and then states that 'our reported rates should be considered lower bounds'. With both directions of error acknowledged, the lower-bound conclusion does not follow unless the overcounting cases were all manually removed or quantified. Please clarify which rates are overcounts, which are undercounts, and why the net claim of 'lower bounds' is warranted for the reported percentages.","section":"§6"}],"minor_comments":[{"comment":"The full-text title is 'WildCode: An Empirical Analysis of Code Generated by ChatGPT', while the arXiv listing is 'WildCode Revisited: A Comprehensive Empirical Study...'. Please unify these and ensure the submitted title matches the manuscript.","section":"Title/Abstract"},{"comment":"The columns 'Labelled by ChatGPT' and 'Labelled by Model' are confusing: for Python, 'Labelled by ChatGPT' equals the total number of code snippets (60,451), and the relationship between these two columns and the final valid-snippet count is not explained. Clarify the denominator used for syntax-error percentages.","section":"Table 2"},{"comment":"The table caption says 'total 177,732 rows', but the sum of the five displayed error categories is 170,516. Either other categories exist and are omitted, or the total is inconsistent.","section":"Table 3"},{"comment":"The text says 1,562 conversations were flagged by OpenGrep and 1,214 were in English, but the preceding Section 4 reports vulnerability counts per category (e.g., hash, SQL, memory, deserialization) that do not obviously sum to 1,562. Please explain whether the 1,562 figure comes from a broader rule set and reconcile the numbers.","section":"§5.4"},{"comment":"Minor typo: 'it’s scale' should be 'its scale'.","section":"§3"}],"recommendation":"major_revision","confidential_remarks":"The user-intent finding and the ReDoS aggregation are the two load-bearing weak points. The former requires validation of the zero-shot classifier or a substantial rewording of the contribution; the latter is a reproducibility issue that should be fixable with a clearer definition and corrected table. I would be comfortable with publication after these are addressed."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: this is the first study I've seen that applies security scanners to code from actual ChatGPT conversations rather than synthetic prompts. That's a real step forward. The dataset is large, and the authors make it available. The security findings are in line with prior work: a lot of generated code is fragile. The ReDoS number (29.4% of 1,203 regexes flagged) and the C/C++ unsafe-function rate (14.85%) are worth having as real-world data points.\n\nCredit where due: the hallucination detection is careful—they filter against PyPI/NPM and manually review. The authors also acknowledge the limits of regex scanning in Section 6, saying it both overcounts (safe contexts) and undercounts (memory errors). That honesty is good.\n\nNow the soft spots. The user-intent analysis (Section 5.2) is the load-bearing new claim, and it rests entirely on zero-shot classification with bart-large-mnli. There is no validation set, no accuracy, no human agreement. The 'Secure Coding' category gets 21 initial labels out of tens of thousands of coding conversations. If the NLI model misses paraphrases like 'make this safe' or 'is this vulnerable to attacks?', the paper's central behavioral finding falls apart. This needs to be addressed before publication—either classifier validation or manual labeling of a sample.\n\nSecond, the abstract promises an evaluation of correctness, but the paper only does syntax checking. That's a mismatch. Syntax-valid code can be functionally wrong. Call it what it is.\n\nThird, Table 2 is confusing if not wrong. The 'Labelled by ChatGPT' column duplicates the snippet counts from Table 1, while 'Labelled by Model' exceeds the total for C/C++ (21,510 vs 7,526). The valid-code numbers only make sense using the 'Labelled by Model' column. This needs a rewrite.\n\nFourth, the abstract says 'proof' while the body says 'evidence.' Keep the softer word.\n\nOverall: the core security result is credible and useful. The user-intent result is novel but currently unsupported. This paper deserves a serious referee and likely major revision. I'd send it to review.","headline":"Real-world WildChat data confirms LLM-generated code is often insecure, but the paper's distinctive user-intent claim rests on an unvalidated classifier.","tokens_in":14672,"tokens_out":7194,"would_cite":true,"duration_ms":58116,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Real-world ChatGPT code is frequently insecure, and users rarely ask for secure code — even when the model produces vulnerable or hallucinated output.","keywords":["LLM-generated code","software security","ChatGPT","real-world conversations","ReDoS vulnerabilities","code hallucinations","user intent","empirical study"],"falsifier":"Randomly sample 500 English coding conversations, have two humans label the initial and follow-up queries for security intent, and compare the human 'secure coding' rate with the 21/34,478 classifier rate; a significantly higher human rate would falsify the paper's user-awareness claim. Alternatively, a data-flow analysis showing most flagged regexes never handle untrusted input would falsify the ReDoS prevalence claim.","tokens_in":13849,"feed_emoji":"🛡️","tokens_out":6341,"duration_ms":50731,"temperature":0.7,"pith_summary":"The paper analyzes 82,843 authentic ChatGPT conversations that contain generated code, and measures how often that code is insecure. It finds that roughly 29% of regular expressions are flagged as ReDoS-vulnerable, 14.85% of C/C++ snippets use unsafe memory functions, and about 14.4% of Python imports are hallucinated package names. A parallel analysis of user intent finds that security is essentially absent from user queries: fewer than two dozen initial requests mention secure coding. The paper's central claim is that synthetic studies were right — LLM-generated code is frequently insecure in real interactions — and that users do not compensate by asking for security.","feed_headline":"One in three ChatGPT regexes is DoS-vulnerable","feed_subtitle":"A study of 82,843 real ChatGPT interactions finds frequent security weaknesses and almost no user demand for secure code.","key_machinery":"The construction pipeline that turns real conversation logs into measurable outcomes: extract code blocks from 82,843 conversations that contain AI-generated code, validate programming-language labels with language-specific compilers and linters, apply 648 regex-based security rules mapped to CWE entries, run four regular-expression denial-of-service detectors, filter import statements against package registries, and classify user intents with a zero-shot model. This pipeline is what converts raw conversation logs into the reported vulnerability rates and intent distributions.","core_discovery":"The paper's central claim is that, at scale, code generated by ChatGPT during real user interactions exhibits frequent, exploitable security weaknesses: 20.61% of conversations involving hash functions triggered weakness rules (mostly MD5/SHA1 or unauthenticated crypto), 14.85% of C/C++ snippets used unsafe memory functions, about 29.4% of 1,203 regexes were flagged as ReDoS-vulnerable by static detectors, and 14.4% of Python imports were hallucinated. The same analysis of the English-language subset finds that users almost never request security: the 'Secure Coding' intent category received only 21 initial labels out of tens of thousands of conversations, and when code was buggy users rarel","pith_inferences":["The user-intent result depends on an unvalidated zero-shot classifier; a human audit of a sample of queries could show the true 'secure coding' rate is higher than 0.06 percent, so the 'users don't ask' finding should be treated as provisional.","Package hallucination names that appear rarely could be pre-registered by attackers as malicious packages, turning a correctness bug into a supply-chain attack vector.","The paper's ReDoS rate may overcount because regexes that never process untrusted input are safe; a follow-up study that traces data flow from user input to the regex would bound the true exploitability rate.","If newer models or domain-specific tools are tested with the same pipeline, the difference in vulnerability rates would reveal whether the problem is model capability or user behavior."],"forward_implications":["Copying ChatGPT code into production without review will import the measured vulnerability rates — roughly one ReDoS-flagged regex in three, and one in seven Python imports hallucinated in the affected snippets.","Prompting users to ask for security is unlikely to work: even users who start with a secure-coding intent rarely sustain the topic in follow-ups.","The memory-safety findings imply that even short C/C++ fragments are risky; assembling larger programs from separately generated fragments may expose use-after-free and double-free bugs that don't appear in small snippets.","Because users won't ask for security, the burden shifts to automated post-generation scanning and default-safe model behavior."],"fun_headline_variants":["Real ChatGPT regexes: 29.4% ReDoS-vulnerable","Only 21 security asks in thousands of ChatGPT chats","One in five ChatGPT hash chats uses weak crypto","14.85% of real C/C++ ChatGPT snippets unsafe memory","Hallucinated Python imports: 14.4% of ChatGPT code"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The claim that users rarely ask for security rests on an unvalidated zero-shot classifier; if the classifier misses security-related phrasings, the near-zero 'Secure Coding' rate could be a measurement artifact rather than a true reflection of user intent.","fun_headline_variants_meta":{"raw":{"variants":["Real ChatGPT regexes: 29.4% ReDoS-vulnerable","Only 21 security asks in thousands of ChatGPT chats","One in five ChatGPT hash chats uses weak crypto","14.85% of real C/C++ ChatGPT snippets unsafe memory","Hallucinated Python imports: 14.4% of ChatGPT code"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000273,"raw_usage":{"total_tokens":1451,"prompt_tokens":703,"completion_tokens":748,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":447,"completion_tokens_details":{"reasoning_tokens":661}},"tokens_in":447,"tokens_out":748,"duration_ms":6381,"temperature":1.0,"reasoning_tokens":661,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T18:36:59.191475+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Randomly sample 500 English coding conversations, have two humans label the initial and follow-up queries for security intent, and compare the human 'secure coding' rate with the 21/34,478 classifier rate; a significantly higher human rate would falsify the paper's user-awareness claim. Alternatively, a data-flow analysis showing most flagged regexes never handle untrusted input would falsify the ReDoS prevalence claim.","supporting_citations":[],"review_version":1}