Pith. sign in

REVIEW 4 major objections 6 minor 30 references

Comparative Analysis of Carbon Footprint in Manual vs. LLM-Assisted Code Development

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read LLM-assisted coding is estimated to emit about 33 times the carbon of manual coding per task.

desk verdict An honest, fully transparent first attempt at comparing manual vs. LLM-assisted coding carbon footprints, but the 32.72x headline is an arithmetic product of two unvalidated constants, not a measured result. read the letter →

arxiv 2505.04521 v1 pith:SRUVWMWY submitted 2025-05-07 cs.SE

Kuen Sum Cheung , Mayuri Kaul , Gunel Jahangirova , Mohammad Reza Mousavi , Eric Zie This is my paper · ORCID
classification cs.SE
keywords carbonfootprintLLM-assistedsoftwaredevelopmentcodegenerationenergyconsumptionGPT-4sustainabilitycompetitiveprogrammingempiricalengineering
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that using a large language model as a coding assistant is not carbon-cheaper than writing code by hand. On twelve competitive-programming tasks from the Codeforces platform, the authors compare the estimated end-to-end energy of a human writing Python with that of a GPT-4-based loop of generate, test, and repair. They report that the LLM-assisted approach uses on average 32.72 times more energy and carbon per task, with the ratio ranging from about 20 to 44. They also find a strong correlation between task difficulty and the size of the gap. If true, the result matters because AI-assisted coding is often promoted as a productivity gain, and the environmental cost of that gain would need to be priced in.

What carries the argument

The argument is carried by the ratio of two energy estimates: $CF_{\text{LLM}} / CF_{\text{manual}}$. The manual side multiplies laptop power ($p_{\text{laptop}}=4.075$ W in the base case) by Codeforces submission times, with debugging and testing added through fixed fractions from prior studies. The LLM side is dominated by a fixed per-query energy of $0.011$ kWh, composed of $0.0022$ kWh for inference and $0.0088$ kWh for amortized training, multiplied by the number of queries the repair loop uses. Because query energy is nearly constant per query while manual energy scales with time, the reported ratio is set largely by these two constants rather than by measured execution.

What would settle it

Measure actual wall-plug energy for both the laptop and the LLM server while solving the same twelve tasks, and recompute the ratio with the measured per-query energy and a realistic laptop power draw; if the measured query energy is well below 0.011 kWh or laptop power is above a few watts, the reported ratio will not reproduce.

Watch

Extended reading notes

Core claim

The central claim is that the end-to-end carbon footprint of LLM-assisted code development exceeds manual development for every task in the sample, by a mean factor of 32.72 with standard deviation 8.41, and that the gap grows with task complexity. The authors model the manual footprint as coding plus debugging plus testing energy, using Codeforces submission times as a proxy for effort. They model the LLM footprint as query energy plus the energy of human insight and repair, with query energy contributing 98.68% of the total. They report a Pearson correlation of 0.890 with a p-value of 0.00011 between task difficulty and the carbon-footprint difference, and interpret this as evidence that LLM assistance is especially costly on harder tasks.

Load-bearing premise

The entire ratio rests on the fixed assumptions that one LLM query costs 0.011 kWh and that coding on a laptop draws only 4.075 W; if either constant is off, especially if real laptop power is 20 to 40 W, the 32.72 times result shrinks or flips.

Editorial extensions

If this is right

  • If the result holds, AI-assisted coding on this kind of task is roughly 20 to 44 times more carbon-intensive per completed task than manual coding, before any productivity savings are counted.
  • Because query energy makes up 98.68% of the LLM footprint, reducing the number of queries per task, or using a cheaper model for easy tasks, would cut the footprint almost linearly.
  • The correlation with task complexity implies that as tasks get harder, the environmental penalty of LLM assistance grows; the authors recommend decomposing complex tasks into smaller subtasks, estimating that this could reduce the gap by up to 30%.
  • For organizations tracking software carbon budgets, the result implies that LLM-based coding should be used selectively rather than as a default for all tasks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The headline ratio is extremely sensitive to the 0.011 kWh per-query constant and the 4.075 W laptop power; using a realistic laptop draw of 20 to 40 W, or dropping the amortized training component from the per-query cost, would shrink the ratio to single digits or reverse it, and the paper itself notes that no power meters were used.
  • The task sample is competitive-programming problems solved in Python, not typical business software, so the ratio for longer, interactive, or more ambiguous real-world tasks could differ because human effort and query counts scale differently.
  • A direct extension would be to run the same twelve tasks with wall-plug energy meters and several different LLMs, varying the per-query energy cost, to map where the crossover between manual and assisted coding lies.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper compares the carbon footprint of manual software development (using Codeforces contest data as a proxy for developer effort) to an LLM-assisted process in which GPT-4 generates and iteratively fixes code for the same tasks. The authors estimate the end-to-end energy of both processes and report that the LLM-assisted approach has on average 32.72 times the carbon footprint of the manual approach, and that the difference correlates with task complexity (Pearson r=0.890). The paper also proposes best practices for reducing LLM-related emissions.

Significance. If the result held, it would be a useful contribution to the growing body of work on the environmental cost of AI-assisted software development, and the replication package would support follow-up studies. The paper is transparent about its formulas, data tables, and assumptions, which makes the identified weaknesses tractable. However, the headline ratio is not robust to the two most influential constants, and the comparison is not outcome-matched for half the tasks, so the significance as a measured result is currently low. The study does provide a structured, reproducible methodology that could be adapted if the underlying estimates are improved.

major comments (4)
  1. [§5.3] The per-query energy constant of 0.011 kWh (0.0022 kWh inference plus 0.0088 kWh training amortization) is the dominant driver of the LLM result, since query energy constitutes 98.68% of LLM CF (§6.1). The training amortization is computed by dividing 50 GWh of training energy by 5.68 Giga queries, but Section 7 acknowledges that GPT-4's lifetime query count is unknown and that no power meters or energy profiling tools were used. A single error in this constant scales the headline ratio almost linearly, yet no sensitivity analysis is provided. Plausible alternative per-query energy values from the literature (e.g., 0.001-0.005 kWh) would reduce the ratio from 32.72 to single digits or below. The paper should present the 32.72x result as conditional on this unvalidated constant and include a sensitivity analysis across published estimates.
  2. [§5.1] The manual arm's power draw is fixed at p_laptop = 4.075W, the average of two laptop life-cycle assessment values (Asus 3.59W, Dell 4.62W). These LCA figures represent average power over the product lifetime, including off and idle states, and are not a valid proxy for the power drawn while a developer is actively coding. Realistic active coding power on a standard laptop is on the order of 20-40W. Replacing 4.075W with 30W alone would increase the manual CF by a factor of about 7.4, reducing the mean ratio from 32.72 to roughly 4.4 even before any change to the query energy constant. The authors need to justify p_laptop with an active-coding power source or demonstrate that the conclusion is invariant to a realistic range of values.
  3. [§6.1 / Table 2] For 6 of the 12 tasks, TPAH (percentage of test cases passed after human insight) is below 100%, and for two tasks it is 0%. This means the LLM-assisted process did not produce a fully working solution for half the tasks, yet these tasks are included in the aggregate ratio of 32.72. The comparison is therefore not between two processes that achieve the same outcome. The research question RQ1 requires both approaches to be evaluated on equivalent outcomes; otherwise one is comparing successful manual development to a partially failing LLM-assisted process. The authors should either restrict the ratio computation to tasks where both arms reach 100% test pass, or report a separate analysis for task success rates and explicitly discuss the implications for the comparison.
  4. [§6.2] The reported correlation of 0.890 between task complexity and the CF difference is structurally dominated by the LLM term. Since query energy is 98.68% of LLM CF (§6.1) and the manual term is negligible in comparison, the 'difference' is approximately equal to the LLM CF. Thus the Pearson and Spearman tests largely measure whether more complex tasks lead to more LLM queries, which is also constrained by the experimental cap of 1-8 queries (§4). The paper should clarify that the correlation does not provide independent evidence about the 'gap' between approaches; it is essentially a property of the query budget.
minor comments (6)
  1. [Abstract] "32.72 higher carbon footprint" should read "32.72 times higher carbon footprint" or "a factor of 32.72."
  2. [§5.3] The unit 'kwh' appears several times (e.g., '0.011 kwh'); use the standard 'kWh' for consistency.
  3. [§6.1] The sentence "the LLM-based approach has at least 19.92 times and at most 43.81 times more carbon footprint (resp. energy consumption)" is awkward; consider phrasing as "by factors of 19.92 to 43.81."
  4. [§6.3] The claim of "up to 30% based on our observed data" is not substantiated; specify which metric is reduced by 30% and where in the data this appears.
  5. [§5.2] The use of Codeforces relative submission time as a proxy for time spent on a task assumes that participants work continuously from contest start; this is not discussed in the threats to validity.
  6. [References] The inference energy value is sourced from a blog post [19]; the authors should seek peer-reviewed sources for per-query inference energy, especially since this constant is so influential.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the headline ratio and the RQ2 correlation are arithmetic or statistical summaries of empirical query counts and stated constants; the paper's acknowledged limitations are validity threats, not circularity.

full rationale

I walked the claimed derivation chain. The manual carbon footprint is computed as p_laptop × MTS plus small testing/debugging terms (Eq. 1–3, Sec. 5.2), and the LLM-assisted carbon footprint is computed as a fixed per-query energy constant (0.011 kWh, Sec. 5.3) times observed query counts plus small human-insight terms. The headline 32.72× ratio is the mean of per-task ratios obtained by ordinary arithmetic from these inputs; no parameter is fitted to the outcome that is then presented as a prediction. The RQ2 correlation is between MTS and (LLM CF − manual CF); because manual CF is only a few percent of LLM CF, the high positive Pearson coefficient reflects the empirical relationship between task time and observed query counts rather than a tautology. The paper itself flags the fragile inputs: 'No actual energy profiling tools or power meters were used to measure the precise energy consumption' (Sec. 5.3) and 'the total number of queries GPT-4 will handle over its lifetime is unknown but we estimated it using the lifetime of GPT-3' (Sec. 7). These are significant validity and robustness limitations, but they do not make the derivation circular. There is no self-citation chain, no imported uniqueness theorem, and no fitted parameter renamed as a prediction. Therefore no circular step is established.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The study contributes measurement framing, but nearly every number in the ratio is an assumed constant. The only genuinely empirical inputs are Codeforces submission times, runtimes, and LLM query counts; the query counts are then multiplied by an assumed per-query energy to produce the headline.

free parameters (6)
  • per_query_energy = 0.011 kWh (0.0022 inference + 0.0088 training)
    Used for every GPT-4 query in Section 5.3; 0.0022 from Ludvigsen [19], 0.0088 computed by authors from 50 GWh divided by 5.68 Giga queries with no cited source; dominates 98.68% of LLM carbon footprint.
  • p_laptop = 4.075 W
    Average of Asus 3.59W and Dell 4.62W lifetime average power reports; used for all manual coding and human insight/add-functionality time. This is far below typical active laptop draw and strongly inflates the manual-to-LLM ratio.
  • carbon_intensity = 217 g CO2e/kWh
    UK Nowtricity 2023 average; location-sensitive but reasonable as an input.
  • debugging_fraction = 0.42 of total development time
    Stripe [24]; combined with Ko et al. 0.1 run fraction to estimate debugging energy consumption.
  • understanding_fraction = 0.38 of total time
    Minelli et al. [22]; used to estimate human insight time.
  • read_extend_fractions = 0.2 + 0.2 of debugging time
    Ko et al. [17]; used for estimated time to add missing functionalities.
assumptions (5)
  • domain assumption Codeforces relative submission time, filtered to sequential completions and outlier-excluded, estimates time spent per task.
    Section 5.2; load-bearing for all manual CEC, DEC, and TEC estimates.
  • domain assumption Laptop power draw during manual coding is a constant 4.075 W.
    Section 5.1; if actual active power is higher, manual carbon footprint rises and the 32.72x ratio falls.
  • ad hoc to paper Every GPT-4 query consumes the same 0.011 kWh regardless of prompt length, output length, or task difficulty.
    Section 5.3; no evidence that GPT-4 query energy is constant or equal to ChatGPT or GPT-3 estimates.
  • ad hoc to paper GPT-4 training energy can be amortized using GPT-3's 50 GWh and 5.68 Giga lifetime queries.
    Section 5.3; the paper itself notes the lifetime is unknown and estimates using GPT-3.
  • domain assumption Debugging and understanding time fractions from general software studies transfer to Codeforces competitive programming.
    Section 5.2; used to estimate debugging and human-insight time without direct measurement.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Comparative Analysis of Carbon Footprint in Manual vs. LLM-Assisted Code Development." pith.science (2026). https://pith.science/paper/SRUVWMWY

@misc{pith2026250504521,
  author       = {Pith},
  title        = {Pith review of: Comparative Analysis of Carbon Footprint in Manual vs. LLM-Assisted Code Development},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SRUVWMWY}},
  note         = {Machine review of arXiv:2505.04521}
}
read the original abstract

Large Language Models (LLM) have significantly transformed various domains, including software development. These models assist programmers in generating code, potentially increasing productivity and efficiency. However, the environmental impact of utilising these AI models is substantial, given their high energy consumption during both training and inference stages. This research aims to compare the energy consumption of manual software development versus an LLM-assisted approach, using Codeforces as a simulation platform for software development. The goal is to quantify the environmental impact and propose strategies for minimising the carbon footprint of using LLM in software development. Our results show that the LLM-assisted code generation leads on average to 32.72 higher carbon footprint than the manual one. Moreover, there is a significant correlation between task complexity and the difference in the carbon footprint of the two approaches.

Figures

Figures reproduced from arXiv: 2505.04521 by the authors.

Figure 1
Figure 1. One Iteration of the LLM Experiment 5.2 Carbon Footprint for Manual Approach In this subsection, our goal is to estimate the carbon footprint of the manual code generation for each task. We calculate the total energy consumption as the sum of three distinct parts: Coding Energy Consumption (CEC), Debugging Energy Consumption (DEC) and Testing Energy Consumption (TEC). Coding Energy Consumption (CEC): This metric ref… view at source ↗
Figure 2
Figure 2. Comparison of Task Complexity and Carbon Footprint [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 26 canonical work pages

  1. [1]

    https://codeforces.com/

    Codeforces. https://codeforces.com/

    work page

  2. [2]

    https://www.nowtricity.com/country/ united-kingdom/

    Current emissions in united kingdom. https://www.nowtricity.com/country/ united-kingdom/

    work page

  3. [3]

    https://www.ipcc.ch/report/ar5/wg3/

    Fifth Assessment Report (AR5): Climate Change 2014: Mitigation of Climate Change. https://www.ipcc.ch/report/ar5/wg3/

    work page 2014

  4. [4]

    https://www.hackerrank.com/

    HackerRank. https://www.hackerrank.com/

    work page

  5. [5]

    https://www.delltechnologies

    LCA Latitude 7300 Anniversary Edition. https://www.delltechnologies. com/asset/en-us/products/laptops-and-2-in-1s/technical-support/full-lca- latitude7300-anniversary-edition.pdf

    work page

  6. [6]

    https://leetcode.com/problemset/

    LeetCode. https://leetcode.com/problemset/

    work page

  7. [7]

    https://unece.org/sites/ default/files/2022-04/LCA_3_FINAL%20March%202022.pdf

    Life Cycle Assessment of electricity generation options. https://unece.org/sites/ default/files/2022-04/LCA_3_FINAL%20March%202022.pdf

    work page 2022

  8. [8]

    https://esg.asus.com/english/file/PEP_ Notebook_C423.pdf

    Product Carbon Footprint Report. https://esg.asus.com/english/file/PEP_ Notebook_C423.pdf

    work page

Show all 30 references
  1. [9]

    https://github.com/HayatoKanee/Green-LLM, 2025

    Replication package. https://github.com/HayatoKanee/Green-LLM, 2025

    2025

  2. [10]

    Green software process factors: A qualitative study

    Siti Rohana Ahmad Ibrahim, Jamaiah Yahaya, and Hasimi Sallehudin. Green software process factors: A qualitative study. Sustainability, 14(18), 2022

    2022

  3. [11]

    Generative ai tools in media and entertainment

    Chris Arkenberg. Generative ai tools in media and entertainment. Deloitte Insights, 2023

    2023

  4. [12]

    Requirements engineering for sustainable software systems: a systematic mapping study

    Peter Bambazek, Iris Groher, and Norbert Seyff. Requirements engineering for sustainable software systems: a systematic mapping study. Requirements Engineering, 28(3):481–505, September 2023

    2023

  5. [13]

    Code green: Evaluating the carbon and energy implications of llm integration in software development

    Boris Belchev. Code green: Evaluating the carbon and energy implications of llm integration in software development. Master’s thesis, University of Twente, 2025

    2025

  6. [14]

    Large language model assisted software engineering: Prospects, challenges, and case study

    Lenz Belzner, Thomas Gabor, and Martin Wirsing. Large language model assisted software engineering: Prospects, challenges, and case study. In Bernhard Steffen, editor, Bridging the Gap Between AI and Reality , pages 355–374, Cham, 2024. Springer Nature Switzerland

    2024

  7. [15]

    LLMCarbon: Modeling the end-to-end carbon footprint of large language models

    Ahmad Faiz, Sotaro Kaneda, Ruhan Wang, Rita Chukwunyere Osi, Prateek Sharma, Fan Chen, and Lei Jiang. LLMCarbon: Modeling the end-to-end carbon footprint of large language models. In The Twelfth International Conference on Learning Representations, 2024

    2024

  8. [16]

    Mourão, and Ivan Machado

    Leila Karita, Brunna C. Mourão, and Ivan Machado. Software industry awareness on green and sustainable software engineering: a state-of-the-practice survey. In Proceedings of the XXXIII Brazilian Symposium on Software Engineering , SBES ’19, page 501–510, New York, NY, USA, 20...

    2019

  9. [17]

    Ko, Brad A

    Andrew J. Ko, Brad A. Myers, Michael J. Coblenz, and Htet Htet Aung. An exploratory study of how developers seek, relate, and collect relevant information during software maintenance tasks. IEEE Transactions on Software Engineering , 32(12):976, 2006

    2006

  10. [18]

    Sustainable software engineering: Visions and perspectives beyond energy efficiency

    Christoph König. Sustainable software engineering: Visions and perspectives beyond energy efficiency. In Proceedings of the 2024 IEEE/ACM 46th International Conference on Software Engineering: Companion Proceedings , ICSE-Companion ’24, page 231–233, New York, NY, USA, 2024. A...

    2024

  11. [19]

    ChatGPT’s Energy Use per Query

    Kasper Groes Albin Ludvigsen. ChatGPT’s Energy Use per Query. https:// towardsdatascience.com/chatgpts-energy-use-per-query-9383b8654487, 2023

    2023

  12. [20]

    Large language models: Their success and impact

    Spyros Makridakis, Fotios Petropoulos, and Yanfei Kang. Large language models: Their success and impact. Forecasting, 5(3):536–549, 2023

    2023

  13. [21]

    Sustainability is stratified: Toward a better theory of sustainable software engineering

    Sean McGuire, Erin Schultz, Bimpe Ayoola, and Paul Ralph. Sustainability is stratified: Toward a better theory of sustainable software engineering. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) , pages 1996–2008, 2023

    2023

  14. [22]

    I know what you did last summer - an investigation of how developers spend their time

    Roberto Minelli, Andrea Mocci, and Michele Lanza. I know what you did last summer - an investigation of how developers spend their time. In 2015 IEEE 23rd International Conference on Program Comprehension , pages 25–35, 2015

    2015

  15. [23]

    CodeCarbon: Estimate and Track Carbon Emissions from Machine Learning Computing

    Victor Schmidt, Karan Goyal, Akhilesh Deepak Joshi, Benjamin Feld, Lyndon Conell, Nikolas Laskaris, Daphne Blank, James Wilson, Sam Friedler, and Sasha Luccioni. CodeCarbon: Estimate and Track Carbon Emissions from Machine Learning Computing. https://doi.org/10.5281/zenodo.465...

    doi

  16. [24]

    The developer coefficient

    Stripe. The developer coefficient. https://stripe.com/files/reports/the-developer- coefficient.pdf, 2018

    2018

  17. [25]

    Models of sustainable software: A scoping review

    Jakub Swacha. Models of sustainable software: A scoping review. Sustainability, 14(1), 2022

    2022

  18. [26]

    Learn to code sustainably: An empirical study on LLM-based green code genera- tion

    Tina Vartziotis, Ippolyti Dellatolas, George Dasoulas, Maximilian Schmidt, Flo- rian Schneider, Tim Hoffmann, Sotirios Kotsopoulos, and Michael Keckeisen. Learn to code sustainably: An empirical study on LLM-based green code genera- tion. arXiv preprint arXiv:2403.03344, 2024

    2024 arXiv

  19. [27]

    Carbon footprint evaluation of code generation through as a service

    Tina Vartziotis, Maximilian Schmidt, George Dasoulas, Ippolyti Dellatolas, Ste- fano Attademo, Viet Dung Le, Anke Wiechmann, Tim Hoffmann, Michael Keck- eisen, and Sotirios Kotsopoulos. Carbon footprint evaluation of code generation through as a service. In 2024 Stuttgart Inte...

    2024

  20. [28]

    Venters, Rafael Capilla, Elisa Yumi Nakagawa, Stefanie Betz, Birgit Penzenstadler, Tom Crick, and Ian Brooks

    Colin C. Venters, Rafael Capilla, Elisa Yumi Nakagawa, Stefanie Betz, Birgit Penzenstadler, Tom Crick, and Ian Brooks. Sustainable software engineering: Reflections on advances in research and practice. Inf. Softw. Technol., 164(C), December 2023

    2023

  21. [29]

    Exploring assessment criteria for sustainable software engineering processes

    Michael Wahler, Norbert Seyff, and Maria Susana Soriano Ramirez. Exploring assessment criteria for sustainable software engineering processes. InProceedings of the 46th International Conference on Software Engineering: Software Engineering in Society, ICSE-SEIS’24, page 107–11...

    2024

  22. [30]

    Large language models for software engineering: A systematic literature review

    Xinyi Hou, Yanjie Zhao, Yue Liu, Zhou Yang, Kailong Wang, Li Li, Xiapu Luo, David Lo, John Grundy, and Haoyu Wang. Large language models for software engineering: A systematic literature review. arXiv preprint arXiv:2308.10620v5, 2023

    2023 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.