{"id":"4fb37d37-c05f-40dc-a4ab-bbd93a01aa57","arxiv_id":"2606.26959","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"Codex usage data from 2025-2026 show fivefold growth in active users, tenfold rise in complex-task requests, and 13-50x increases in monthly output tokens for legal and research roles.","lead":"The paper examines usage logs from OpenAI's Codex tool across internal and external users to document rapid growth in agentic AI adoption. The data show fivefold increases in active users, rising task complexity, and large gains in output tokens, particularly inside the company.","discovery_kind":"unclear","skeptic_critique":{"model":"grok-4.3","headline":"Unvalidated automated pipeline for estimating human task completion time and attributing tokens underpins all growth statistics","rationale":"The reader's weakest assumption correctly isolates the pipeline as the single point on which every quantitative claim rests. Because the manuscript is descriptive rather than causal and the initial review was abstract-only, this measurement gap is sufficient to keep the verdict at UNVERDICTED; no additional internal inconsistency is visible from the supplied material.","tokens_in":1852,"tokens_out":311,"duration_ms":17926,"concrete_test":"Draw a stratified random sample of 200 requests (50 per user population per month) from the study window; obtain independent human estimates of completion time and Codex/ChatGPT attribution from two blinded raters; measure agreement with pipeline labels. If Cohen's kappa on complexity class <0.65 or mean token attribution discrepancy >15%, the reported growth rates are sensitive to measurement error.","verdict_should_be":"UNCHANGED","load_bearing_attack":"All reported figures (fivefold user growth, nearly tenfold rise in >8-hour tasks, 13x/50x median token increases) are generated by a single automated pipeline that classifies request complexity via estimated human completion time and partitions output tokens between Codex and ChatGPT. The abstract provides no validation data, inter-rater reliability, or sensitivity checks on the time-estimation model, nor on how the pipeline handles novel agentic workflows, concurrent agents, or internal tool interactions. Systematic bias in either component would directly scale the headline multipliers.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper analyzes usage logs from OpenAI's Codex tool across external personal, external organizational, and internal OpenAI user populations. It reports rapid growth in agentic AI adoption (more than fivefold increase in active users in H1 2026), rising workflow sophistication (over 10% managing three or more concurrent agents weekly; 26.6% using skills), increased task complexity (nearly tenfold rise in share of users submitting >8-hour tasks), and large output growth (13x median monthly tokens for legal roles and 50x for researchers within OpenAI), all derived from an automated privacy-protecting pipeline that classifies tasks by estimated human completion time and attributes tokens between Codex and ChatGPT.","tokens_in":1962,"tokens_out":509,"duration_ms":15726,"significance":"If the pipeline measurements prove reliable, the manuscript supplies large-scale descriptive evidence on the diffusion of agentic AI, documenting shifts away from ChatGPT, uneven adoption, and proxies for productivity gains. Such data could inform models of technology adoption and labor reorganization in economics. The internal OpenAI population offers a useful benchmark, but the absence of validation or external benchmarks limits the strength of the claims.","major_comments":[{"comment":"Methods: The automated pipeline that estimates human task completion time and partitions output tokens between Codex and ChatGPT underpins every headline statistic (fivefold user growth, tenfold complexity increase, 13x/50x token multipliers), yet the manuscript reports no validation data, inter-rater reliability metrics, sensitivity checks, or handling of novel agentic workflows and concurrent agents.","section":"Methods"},{"comment":"Results: No sample sizes, confidence intervals, or robustness checks accompany the reported growth rates or median token increases, preventing assessment of precision or sensitivity to pipeline assumptions.","section":"Results"},{"comment":"Data section: Potential selection bias in the three user populations and measurement error in the internal OpenAI logs are not addressed, even though the pipeline is the sole source for all complexity and attribution classifications.","section":"Data"}],"minor_comments":[{"comment":"Abstract: Dates (November 2025–June 2026) are referenced without stating the exact observation window or data-collection start date.","section":"Abstract"}],"recommendation":"major_revision","confidential_remarks":"The entire analysis rests on proprietary OpenAI internal data with no external replication path; this raises standard concerns about replicability and disclosure of potential conflicts."},"author_rebuttal":{"model":"grok-4.3","summary":"Thank you for the opportunity to respond to the referee's report. The referee correctly identifies that the pipeline is central to our findings and that additional details on methods, results precision, and data limitations would improve the manuscript. We address each point below and propose revisions accordingly. We note that certain validation aspects cannot be provided due to the privacy design of our data pipeline.","responses":[{"response":"We acknowledge the importance of validating the pipeline. Unfortunately, the privacy-protecting nature of the pipeline prevents us from retaining or accessing data necessary for external validation or inter-rater reliability studies. We will, however, add sensitivity checks to the revised manuscript and provide more explicit discussion of how the pipeline handles concurrent agents and novel workflows based on the classification logic.","revision_made":"partial","referee_comment":"[Methods] Methods: The automated pipeline that estimates human task completion time and partitions output tokens between Codex and ChatGPT underpins every headline statistic (fivefold user growth, tenfold complexity increase, 13x/50x token multipliers), yet the manuscript reports no validation data, inter-rater reliability metrics, sensitivity checks, or handling of novel agentic workflows and concurrent agents."},{"response":"We agree that including sample sizes, confidence intervals, and robustness checks would enhance the presentation of results. In the revised version, we will report the underlying sample sizes for key statistics and include confidence intervals where feasible. We will also conduct and report additional robustness checks regarding the pipeline assumptions.","revision_made":"yes","referee_comment":"[Results] Results: No sample sizes, confidence intervals, or robustness checks accompany the reported growth rates or median token increases, preventing assessment of precision or sensitivity to pipeline assumptions."},{"response":"We will revise the data section to explicitly discuss potential selection biases across the external personal, organizational, and internal populations, as well as any measurement considerations in the internal logs. This will include a more detailed description of the populations and limitations.","revision_made":"yes","referee_comment":"[Data] Data section: Potential selection bias in the three user populations and measurement error in the internal OpenAI logs are not addressed, even though the pipeline is the sole source for all complexity and attribution classifications."}],"tokens_in":1559,"tokens_out":467,"duration_ms":20080,"standing_objections":["Validation data, inter-rater reliability metrics, or external benchmarks for the automated pipeline due to its privacy-protecting design"]},"desk_editor":{"model":"grok-4.3","letter":"This paper reports usage patterns from OpenAI's Codex across personal accounts, organizational accounts, and internal employees. The new material is the scale of the logs plus the focus on concurrent agents, skills, task complexity measured in human hours, and token output splits. Those specifics on workflow changes and the internal shift away from ChatGPT do not appear in earlier ChatGPT studies.\n\nThe data show clear trends: fivefold user growth, rising share of long tasks, and large token increases for some roles. Raw counts from production logs are a step up from surveys, and the three-population comparison is useful for seeing uneven external adoption.\n\nThe soft spot is the automated pipeline that estimates human task times and attributes tokens. The abstract supplies no validation, sensitivity checks, or sample sizes, so any systematic error in classification would scale all the multipliers directly. Internal OpenAI data also leaves open questions about how representative the patterns are.\n\nThis is for labor and productivity researchers who want early numbers on agentic AI uptake. A reader can extract the descriptive facts while treating the exact growth rates as provisional.\n\nIt deserves peer review. The dataset is novel enough that referees should see the full methods and any robustness work.","headline":"Codex logs give the first big descriptive picture of agentic coding tool spread and output growth, but the headline multipliers rest on an unvalidated automated pipeline.","tokens_in":2493,"tokens_out":321,"would_cite":true,"duration_ms":13063,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Codex usage data shows agentic AI adoption grew more than fivefold in the first half of 2026 with nearly tenfold higher task complexity and sharply rising output tokens.","keywords":["agentic AI","Codex","usage patterns","task complexity","productivity","workflow changes","organizational adoption","output tokens"],"falsifier":"A re-analysis or external audit of the raw request logs that finds the human-time estimates or Codex-versus-ChatGPT token attributions deviate by more than a factor of two from the reported figures on average.","tokens_in":2749,"feed_emoji":"📊","tokens_out":812,"duration_ms":22142,"temperature":0.7,"pith_summary":"The paper examines internal logs from OpenAI's Codex tool to track how agentic AI systems that act on users' behalf are altering work patterns across different groups. It documents rapid growth in active users, especially outside software development, alongside increases in the sophistication of requests and the volume of generated output. A sympathetic reader would care because these trends suggest concrete shifts in daily workflows and potential effects on productivity and job structure as such tools spread.","feed_headline":"Codex data shows agentic AI users grew fivefold in six months","feed_subtitle":"Request complexity rose nearly tenfold and output tokens increased up to 50 times for researchers inside OpenAI.","key_machinery":"An automated privacy-protecting pipeline that classifies Codex requests by estimated human completion time and attributes output tokens across user populations and tools.","core_discovery":"We use an automated, privacy-protecting pipeline to contrast usage across three populations: external personal-account users, external organizational-account users, and workers within OpenAI. Agentic AI usage grew more than fivefold in the first half of 2026, with the most rapid increase outside the initial audience of software developers. Within OpenAI, Codex usage is nearly universal and has largely replaced business usage of ChatGPT. A similar though slower shift occurs outside OpenAI, particularly within organizations. More than 10% of users manage three or more concurrent Codex agents each week, 26.6% use skills for sharing complex workflows, the share of users submitting tasks estimate","pith_inferences":["If the internal patterns generalize, organizations adopting agentic tools may experience measurable gains in task throughput without proportional increases in headcount.","The faster uptake inside OpenAI compared with external groups points to company-level policies or infrastructure as accelerators of the shift.","Rising concurrent agent use and skill sharing could lead to new job roles centered on orchestrating and maintaining multiple AI agents.","Comparing productivity metrics such as project completion rates before and after Codex rollout would test whether the observed token increases translate into net output gains."],"forward_implications":["The number of active Codex users grew more than fivefold in the first half of 2026, fastest outside software developers.","Within OpenAI, Codex usage became nearly universal and replaced most business ChatGPT use.","Over 10 percent of users managed three or more concurrent agents weekly and 26.6 percent used skills to share complex workflows.","The share of users submitting requests for tasks estimated to take more than eight human hours rose nearly tenfold since the start of the year.","Median monthly output tokens rose 13 times for legal roles and 50 times for researchers inside OpenAI between November 2025 and June 2026."],"fun_headline_variants":["Codex data shows agentic AI users grew fivefold in six months","Inside OpenAI Codex replaces ChatGPT for most business tasks","Over 10% of users run three or more Codex agents weekly","Codex task complexity rises nearly tenfold since early 2026","OpenAI researchers produce 50 times more output tokens via Codex"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The automated pipeline correctly classifies tasks by estimated human completion time and accurately attributes output tokens to Codex versus ChatGPT without systematic measurement error or selection bias in the three user populations.","fun_headline_variants_meta":{"raw":{"variants":["Codex data shows agentic AI users grew fivefold in six months","Inside OpenAI Codex replaces ChatGPT for most business tasks","Over 10% of users run three or more Codex agents weekly","Codex task complexity rises nearly tenfold since early 2026","OpenAI researchers produce 50 times more output tokens via Codex"]},"model":"grok-4.3","cost_usd":0.003786,"raw_usage":{"total_tokens":2043,"prompt_tokens":842,"num_sources_used":0,"completion_tokens":87,"cost_in_usd_ticks":37862000,"prompt_tokens_details":{"text_tokens":842,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1114,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":842,"tokens_out":87,"duration_ms":8423,"temperature":1.0,"reasoning_tokens":1114,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-26T01:50:42.385337+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A re-analysis or external audit of the raw request logs that finds the human-time estimates or Codex-versus-ChatGPT token attributions deviate by more than a factor of two from the reported figures on average.","supporting_citations":[],"review_version":1}