{"id":"db867dca-06f1-4433-943c-e0bb960c4f82","arxiv_id":"2608.06640","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"AI-generated C++ in a large enterprise shows concentrated interface/coupling and copy/allocation issues, more explicit loops, and modestly higher compute and memory growth than human-written code; targeted feedback reduces static findings in a small benchmark.","lead":"This paper measures how AI-written C++ behaves in production, using authoring-time tracking across 3.52 million code changes at one large company. It finds a distinct quality profile: more interface and copy overheads, extra compute and memory growth, and more reviewer effort, with targeted feedback partially correcting the pattern.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 5-8% compute-cost claim rests on an unadjusted growth comparison between AI-heavy and human-typed function cohorts; uneven AI adoption across org slices means the gap may reflect cohort mix rather than authorship.","rationale":"The reader's weakest_assumption already flags compute-cohort comparability ('treated as if differences in normalized resource growth reflect code authorship rather than differing task mix'), so there is substantial agreement. However, I identify a more precise and load-bearing location: the compute comparison lacks the organizational/product-area adjustment that the paper applies to its other outcome comparisons, despite the paper itself documenting highly uneven adoption across organizational slices. This makes the abstract's 5-8% compute figure the least secure element of the central claim. The provenance attribution, while imperfect, is substantially stronger because it uses authoring-time byte-level observation; the compute comparison is purely observational and only edit-volume stratified. The intervention (RQ4) is also limited by its 50-function synthetic benchmark and same-taxonomy circularity, but the compute claim is more central to the paper's stated contribution and to the abstract. Since the reader's verdict is already CONDITIONAL and this concern supports that conditional stance, I do not recommend a verdict change. The concrete test—stratify the compute cohorts by organizational slice and report absolute levels—would settle whether the concern lands. If the gap disappears, the paper's headline cost claim should be withdrawn or substantially reframed; if it persists, the conditional verdict could be upgraded toward ACCEPT on the compute dimension.","tokens_in":17592,"tokens_out":7250,"duration_ms":66509,"concrete_test":"Re-run the Section 4.3 compute growth analysis with cohorts stratified by the same coarse organizational slices shown in Figure 3 (e.g., ML/AI vs Core Infrastructure vs Consumer), and additionally report absolute normalized compute cost per function at baseline and at end-of-window, not only growth indices. If the ~5% CPU / ~8% memory growth gap disappears within strata or is driven by high-growth org slices, the authorship interpretation is confounded by product-area growth; if the gap persists within slices with matched task categories, the confound is ruled out.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract's headline downstream claim—that AI-generated code causes a 5-8% increase in compute resource consumption—rests on the longitudinal cohort comparison in Section 4.3 (Figure 4, bottom panels) and on the 'Execution profile shifts' Table 5. The compute comparison measures growth in percent-of-application CPU and memory share for approximately 6k AI-heavy vs 6k human-typed functions, with the only stated adjustment being edit-volume stratification (Section 3.4: 'Our primary specification stratifies these cohorts by their respective edit volumes'). No adjustment for coarse organizational slice, product area, or function task category is reported for this comparison, in contrast to the review/reliability comparisons which announce stratification 'across month, change size, coarse organizational slice, anonymized author or team controls, and new-code status where available.' This asymmetry matters because Section 4.1/Figure 3 shows AI adoption is highly uneven: ML/AI reached 70.19% AI share while other slices plateaued at lower levels. If the AI-heavy function cohort is concentrated in faster-growing product areas, its percent-of-application share can rise even if per-function code efficiency is identical, because the denominator is total application compute for the whole fleet. The model-mix masking (Section 6) adds that the pooled AI signal spans multiple model generations, so even a genuine gap would not identify a stable authorship effect. Thus the causal-sounding '5-8% increase' is not supported by the reported analyses; the observed growth gap is equally consistent with cohort composition differing in task mix and product growth. The static quality profile (RQ2) and review-friction results are less affected, but the headline downstream-cost claim would be removed or downgraded unless the compute comparison is properly adjusted.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This manuscript reports a one-year observational study of AI-generated C++ code in a large enterprise monorepo, using authoring-time byte-level provenance to attribute code to AI versus human authorship. It addresses four research questions: where AI-generated code appears (RQ1), what upstream structural and static issue profiles look like (RQ2), how those profiles relate to review, reliability, and compute outcomes (RQ3), and whether taxonomy-informed prompt feedback reduces targeted static warnings and improves benchmark efficiency (RQ4). The paper's central claims are that AI-generated C++ exhibits a distinct quality profile (larger changes, concentrated interface/coupling and copy/allocation static findings, more explicit loops, less standard-library use) that translates into increased review effort and an approximate 5-8% relative growth gap in compute and memory consumption, and that targeted feedback yields an 11.1% reduction in targeted static findings. The scale of the dataset (3.52 million changes, 10.46 million C++ LOC) and the provenance instrumentation are notable strengths, but the downstream causal claims are not fully supported by the reported analyses.","tokens_in":17842,"tokens_out":6236,"duration_ms":55001,"significance":"If the central claims hold, this is the first large-scale production evidence that AI-generated C++ carries a concentrated, measurable quality and efficiency burden beyond controlled benchmarks. The taxonomy and the provenance-based measurement approach are valuable contributions, and the RQ4 feedback intervention is a practical step toward mitigation. However, the load-bearing downstream claims about review effort and compute cost outrun the current evidence: the compute figures are unadjusted growth-gap comparisons, the review figures are unadjusted raw contrasts with no stratified results shown, and the RQ4 outcome reuses the same analyzer that defined the target categories. The paper is nonetheless a strong descriptive industrial study with an unusually complete observation pipeline, and with careful reanalysis or tightened claims it could become an important reference point for empirical software engineering research.","major_comments":[{"comment":"The abstract's '5-8% increase in compute resource consumption' is a relative growth gap between cohort indices: 1.31x versus 1.25x for compute and 1.36x versus 1.25x for memory, i.e., approximately 5% and 8% relative differences in indexed growth. This comparison is adjusted only for edit volume (as stated in §3.4), with no controls for organizational slice, product area, or function task category. Because §4.1 shows AI adoption is highly uneven across organizational slices (e.g., 70.19% in ML/AI versus lower elsewhere), the observed growth gap may reflect cohort mix rather than per-function code authorship. The sentence 'This confirms that the upstream efficiency issues identified in RQ2 translate into measurable compute and memory overheads at scale' asserts a causal interpretation that this design cannot support. The authors should either present adjusted models for this comparison or explicitly label the figures as descriptive and soften the causal language.","section":"§4.3, Figure 4 bottom panels"},{"comment":"The paper promises 'stratified cohort comparisons' with controls for month, change size, coarse organizational slice, author/team, and new-code status, but the review and reliability results in §4.3 present only unadjusted 'descriptive downstream contrasts' (e.g., 1.92x blocking threads, 1.39x comments, 1.24x reviewer iterations). No adjusted effect sizes, confidence intervals, or model outputs are shown for these outcomes. Given that Table 2 shows AI-generated changes are larger and more new-code-heavy, the raw ratios likely overstate the authorship effect. The claim of 'increased review effort' requires the reported stratified estimates, or the claim should be downgraded to a descriptive association.","section":"§3.4 and §4.3"},{"comment":"The RQ4 evaluation is partially circular: the 50 benchmark functions were selected because they contain at least one target-category finding from the same static taxonomy used to build the intervention prompt, and the primary outcome is the count of those same targeted static findings. The 11.1% reduction therefore partly reflects the prompt's direct acknowledgement of the analyzer's preferred issue types, not necessarily an improvement in general code quality. The authors should validate the intervention with out-of-taxonomy measures (e.g., independent manual review, different analyzers, or broader efficiency metrics) or explicitly restrict the claim to reductions on the analyzer-defined target categories in the results section rather than only in the threats-to-validity discussion.","section":"§4.4, Table 6"},{"comment":"The causal framing of the compute and review comparisons treats AI-heavy versus human-typed function cohorts as if authorship is the only systematic difference. However, the pooled AI signal spans multiple model generations and interaction modes, and model identifiers are masked (§6). Developer editing during authoring can blur the byte-level provenance, as acknowledged in §6. These factors mean the observed differences should be interpreted as specific to the deployment under study, not as a general 'AI effect.' The abstract and conclusion should qualify the claims accordingly, and the authors should report sensitivity analyses that vary the AI-share threshold or restrict to stable sub-cohorts if such analyses are feasible.","section":"§3.2 and §6"}],"minor_comments":[{"comment":"The abstract says '5-8% increase in compute resource consumption,' but the text reports relative growth gaps between cohort indices; the phrase 'relative increase' should appear in the abstract to avoid implying a direct causal increase in absolute resource consumption.","section":"Abstract and §4.3"},{"comment":"The bottom panel labels read 'Compute Growth (2025-05 = 1)' and 'Memory Growth (2025-05 = 1)', but the text says the baseline is 'at the start of our study window' (April 1, 2025). Please align the baseline date in the figure with the study window definition.","section":"Figure 4"},{"comment":"The R_eff score is described as mapping instruction/memory comparison to three values (1, 0.5, 0), but the table reports averages like 0.294; explain how the coarse per-function scores are aggregated and what a score of 0.294 means in practical terms.","section":"§4.4, Table 6"},{"comment":"In Table 6, the baseline row has no R_eff value; the text explains that baseline is the original code, but the table should either include R_eff = 1.0 by definition or an explicit 'not applicable' note.","section":"§4.4"},{"comment":"The phrase 'A intervention' appears at the beginning of §3.1; it should be 'An intervention.'","section":"§3.1 and §4.4"},{"comment":"The row for 'Standard library use' reports a ratio of '~0.4x' with the description 'std::* usages'; the text in §4.2 says 'uses standard-library/API calls 30–40% less often,' which is consistent, but the table should state the exact normalized ratio or range to avoid ambiguity.","section":"§4.2, Table 4"},{"comment":"The taxonomy validation reports '2 annotators' without giving the inter-rater agreement value; the text says the validation records it, but the number is not reported in the manuscript. Please include the agreement statistic or state that it was computed and used for the final mapping.","section":"§3.3"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is based on proprietary industrial data, which limits independent replication but is not itself a reason for rejection. The main concern is that the abstract's causal-sounding claims (5-8% compute increase, increased review effort) are not matched by the reported analyses; the authors should either add the missing adjusted models or explicitly relabel these results as descriptive. The RQ4 circularity is fixable with additional validation or careful language. I would encourage the authors to include the replication package with aggregate statistics and to consider whether the claims can be tightened to what the current design actually supports."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read this one. It is the first study I have seen that follows AI-generated C++ from authoring-time provenance through review, static findings, and function-level compute, at a scale (3.52M changes) that makes the descriptive results matter. The provenance design is the real contribution: byte-level attribution during authoring avoids the usual post hoc detector problem, and the authors are honest about its limits.\n\nThe static profile is the strongest part. The finding that the AI/human issue-rate gap is concentrated in Interface and Coupling Burden (1.15x) and Copy and Allocation Overhead (1.39x) is plausible and actionable, and the supporting taxonomy work—two annotators, validation sample, coverage reporting—is more rigorous than most industry studies. The review-friction results (1.92x blocking threads, 1.39x total comments) are also credible and are exactly what you would expect if AI changes are larger and newer. The lower revert rate is a useful counterpoint.\n\nThe soft spot is the compute claim. The '5-8% increase' in the abstract is not a measured per-function difference; it is a relative growth gap between cohort indices (1.31x vs 1.25x CPU, 1.36x vs 1.25x memory), with only edit-volume stratification reported. The paper's own figures show AI adoption is uneven across org slices, so the growing gap could be cohort mix, task mix, or product growth rather than authorship. The authors mask model identifiers, so the pooled AI signal spans multiple model generations. The stress-test note is right: as reported, the compute comparison cannot support the causal-sounding abstract claim. That said, the authors acknowledge the observational design and the confounding in Section 6, so this is an overstatement rather than a fabrication. They need to either add adjustments (org slice, task category, model generation proxy) or downgrade the claim.\n\nThe 11.1% intervention reduction is real but narrow: 50 synthetic functions, three runs per stage, and the target categories come from the same taxonomy and analyzer used to measure the reduction. It is a proof-of-concept, not a production result. Fine to report, but it should not be read as evidence that prompt feedback will reduce fleet-wide compute.\n\nThe paper deserves a serious referee. The static profile and the provenance method are worth publishing even if the compute claim gets softened. I would accept it with major revisions: fix or reframe the compute analysis, clarify the intervention scope, and release whatever anonymized aggregated data they can.","headline":"First production-scale provenance trace of AI-generated C++: the static quality profile is credible and valuable, but the headline 5-8% compute-cost claim is a confounded growth-gap that should be downgraded.","tokens_in":18515,"tokens_out":2406,"would_cite":true,"duration_ms":20792,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"AI-generated C++ in production carries a concentrated quality burden in coupling and copy overhead, and targeted taxonomy-informed feedback measurably reduces it.","keywords":["AI-generated code","C++","software quality","static analysis","code review","compute efficiency","authoring provenance","empirical software engineering"],"falsifier":"A re-analysis that recomputes AI-versus-human attribution after fully resolving overlapping authoring bytes, then matches AI-heavy and human-typed function cohorts on task type, repository context, model generation, and developer experience, would settle the central claim. If the 5-8% compute and memory gap and the concentrated static-finding gap disappear under that matching, the reported quality profile is an artifact of cohort differences rather than of AI authorship.","tokens_in":17323,"feed_emoji":"⚙️","tokens_out":8815,"duration_ms":77229,"temperature":0.7,"pith_summary":"Using authoring-time provenance on 3.52 million submitted changes and a C++ slice of 10.46 million lines, this paper tries to establish that AI-generated C++ has a distinct, measurable quality profile once it reaches production: more interface and coupling burden, more copy and allocation overhead, and more explicit loops instead of optimized standard-library calls. The paper then argues these upstream patterns show up downstream as extra review effort and a 5-8% increase in normalized compute and memory consumption. It also argues the profile is not a hard model ceiling, because adding targeted, taxonomy-informed feedback to generation cut targeted static-analysis warnings by 11.1% and improved an instruction/memory efficiency score by 31%. If correct, the result gives production teams a concrete, prioritized list of code properties to steer AI assistants away from, and an evaluation standard that goes beyond pass rates and task completion.","feed_headline":"AI-generated C++ costs 5-8% more compute in production","feed_subtitle":"A 3.52M-change study finds coupling and copy overhead, and targeted feedback trims warnings by 11.1%.","key_machinery":"The carrying mechanism is authoring-time provenance: every changed byte is tagged during authoring as AI-generated or human-typed, then projected onto the lines, functions, and static findings of the final submitted snapshot. This byte-level attribution is joined to a three-level static taxonomy, with quality attributes, issue categories, and issue types, that collapses raw static checks into developer-facing mechanisms such as missing-move and include-cleaner. The same provenance feeds function-level compute cohorts, so the paper can connect a line-level property like a hand-written loop instead of a standard-library call to a production outcome like normalized CPU and heap growth. The intervention reuses the taxonomy as prompt content: the coding agent is asked to reimplement functions under base, general-feedback, and category-specific-feedback conditions, with static finding counts and an efficiency score as outcomes.","core_discovery":"The paper's central claim is that AI-generated C++ in a large brownfield production codebase carries a concentrated, patterned quality burden that survives code review and shows up in operating costs. In the submitted code, AI changes are larger and more new-code-heavy, and their excess static-finding rate is concentrated in Interface and Coupling Burden and Copy and Allocation Overhead, which account for 82.21% of the positive absolute rate gap. Source-level measures show AI code authors loops at roughly twice the human rate, uses standard-library or API calls 30-40% less often, and has higher rates of missing-move, use-emplace, and inefficient-map warnings. Downstream, AI changes receive 1.92x more blocking review threads and 1.39x more comments, and deployed AI-heavy functions grow to about 5% more normalized compute and 8% more memory than human-typed functions over the study window. The paper also claims that adding category-specific feedback to a reimplementation prompt reduces targeted static findings by 11.1% and raises the benchmark efficiency score by about 31%, showing the observed weaknesses respond to targeted mitigation.","pith_inferences":["The authors leave implicit that the loop-and-avoid-library pattern is likely a brownfield knowledge problem: generalist models trained largely on public code do not know an enterprise's internal optimized libraries, so the pattern may reflect missing repository knowledge more than model capability.","If the profile replicates, the same method should produce language-specific taxonomies for Rust, Go, and Java; the C++ categories such as missing-move and use-emplace will not transfer directly, but the imperative-versus-library contrast may.","The paper's own null result on review metrics hints that reviewer effort does not intercept these costs, which would make automated, prompt-time or training-time feedback the only scalable mitigation.","Because the pooled AI signal spans multiple model generations and interaction modes, the 5-8% estimate likely mixes older and newer models; a per-model breakdown could show the overhead is partly a legacy-model artifact that later models already avoid."],"forward_implications":["Reviewers and static-analysis tooling can prioritize Interface and Coupling Burden and Copy and Allocation Overhead, which together make up 82.21% of the positive rate gap.","Production evaluation of AI coding assistants should track code past submission: the AI-specific cost appears as review friction and compute growth, not as higher revert rates, which stayed below parity.","Category-specific prompt feedback is a working mitigation, cutting targeted static findings by 11.1% and improving the benchmark efficiency score by 31% in the controlled regeneration setting.","If the compute and memory gap scales with AI share, large organizations adopting AI assistants can expect a persistent 5-8% resource overhead that grows with adoption, making efficiency part of AI sustainability planning.","The results suggest an automated loop where static analysis identifies recurring category-level risks and feeds them back into generation, rather than relying on reviewers to catch localized inefficiencies."],"supporting_citations":[{"why":"Establishes the line-level static-analysis ecosystem whose findings and workflow constraints the study joins with provenance and review data.","marker":"[33]"},{"why":"Establishes the modern-code-review workflow and metrics such as comments and blocking threads used as downstream outcomes.","marker":"[34]"},{"why":"Documents runtime performance costs in C++ code written through an AI code-completion assistant, motivating the production-lifecycle analysis.","marker":"[12]"},{"why":"Identifies the same copy-move, container, and API-selection idioms as warehouse-scale optimization targets, motivating the source-level efficiency measures.","marker":"[22]"},{"why":"Provides the prioritization standard for actionable quality signals in large industrial codebases, used to justify focusing on high-burden categories.","marker":"[17]"},{"why":"Supplies the empirical taxonomy method and error-pattern codebook used to build the three-level static taxonomy.","marker":"[44]"},{"why":"Provides the prior LLM feedback-and-verification approach that the taxonomy-informed feedback intervention builds on as a baseline.","marker":"[26]"},{"why":"Supports the measurement of review outcomes such as comments and resolution latency used in the downstream analysis.","marker":"[13]"},{"why":"Documents runtime memory divergence in LLM-generated code, supporting the compute-and-memory outcome framing.","marker":"[32]"}],"fun_headline_variants":["AI C++ costs 5-8% more compute, 2x review threads","AI C++ flaws: 82% from coupling and copies; compute +5-8%","Feedback trims AI C++ warnings 11.1% and compute cost","AI C++: 2x review threads, 5-8% more compute, fixable"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that byte-level authoring-time provenance, after developers edit and revise their code, still correctly attributes the final submitted and deployed lines to AI versus human authorship, and that the compute cohorts differ in authorship rather than in task mix, model generation, or developer skill.","fun_headline_variants_meta":{"raw":{"variants":["AI C++ costs 5-8% more compute, 2x review threads","AI C++ flaws: 82% from coupling and copies; compute +5-8%","Feedback trims AI C++ warnings 11.1% and compute cost","AI C++: 2x review threads, 5-8% more compute, fixable"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000826,"raw_usage":{"total_tokens":3678,"prompt_tokens":1080,"completion_tokens":2598,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":696,"completion_tokens_details":{"reasoning_tokens":2504}},"tokens_in":696,"tokens_out":2598,"duration_ms":17327,"temperature":1.0,"reasoning_tokens":2504,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T04:10:44.510613+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A re-analysis that recomputes AI-versus-human attribution after fully resolving overlapping authoring bytes, then matches AI-heavy and human-typed function cohorts on task type, repository context, model generation, and developer experience, would settle the central claim. If the 5-8% compute and memory gap and the concentrated static-finding gap disappear under that matching, the reported quality profile is an artifact of cohort differences rather than of AI authorship.","supporting_citations":[{"cited_title":"Sadowski, E","cited_arxiv_id":null,"evidence_quote":"Establishes the modern-code-review workflow and metrics such as comments and blocking threads used as downstream outcomes."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the empirical taxonomy method and error-pattern codebook used to build the three-level static taxonomy."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Documents runtime memory divergence in LLM-generated code, supporting the compute-and-memory outcome framing."}],"review_version":1}