Do Generative AI Tools Ensure Green Code? An Investigative Study

Adam P. Burden; Rohit Mehra; Samarth Sikand; Sanjay Podder; Vibhu Saujanya Sharma; Vikrant Kaulgud

REVIEW 4 major objections 5 minor 26 references

Do Generative AI Tools Ensure Green Code? An Investigative Study

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that ChatGPT, BARD, and Copilot default to non-green code: no tool gave the optimal Java I/O solution, and Copilot offered a non-green option for five of six rules.

desk verdict A small but honest first empirical look at whether AI code tools follow green coding rules; the pattern-based evidence is plausible, but the energy-impact claim outruns the measurements. read the letter →

arxiv 2506.08790 v1 pith:V7BZN7D2 submitted 2025-06-10 cs.SE cs.AIcs.CY

Samarth Sikand , Rohit Mehra , Vibhu Saujanya Sharma , Vikrant Kaulgud , Sanjay Podder , Adam P. Burden This is my paper

classification cs.SEcs.AIcs.CY

keywords greencodegenerativeAIsoftwaresustainabilityenergy-efficientcodinggenerationtoolsChatGPTCopilotBARD

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper investigates whether code produced by ChatGPT, BARD, and Copilot follows known sustainable coding practices. It selects six green-coding rules, five from an industrial green-IT rule catalog and one based on measured energy behavior of Java I/O APIs, crafts two natural-language prompts per rule, and checks whether the tools' default output uses the recommended pattern. The reported result is that none of the tools produced the most energy-efficient Java I/O solution, and Copilot gave at least one non-green suggestion for five of six rules, with ChatGPT and BARD also missing rules. If this holds, developers who accept tool output as-is will frequently commit code that is less energy-efficient than available alternatives, and the growing share of AI-generated code would carry that overhead at scale.

What carries the argument

The carrying mechanism is a rule-to-prompt evaluation grid: six sustainable-coding rules, each represented by two natural-language prompts, run against each of the three tools, with outputs manually inspected for the presence or absence of the prescribed green pattern. ChatGPT and BARD are scored by whether their single conversational reply contains the pattern, while Copilot's default panel of up to ten suggestions yields a percentage of green suggestions, such as 5 of 10 for the JavaScript loop rule. This grid converts greenness into an observable yes-or-no signal per rule, which is what lets the paper speak about default behavior.

What would settle it

Run the same six prompt pairs against current versions of the three tools, execute each emitted solution on representative workloads while metering processor and memory energy, and compare pattern-adherent versus pattern-lacking outputs; if the non-adherent solutions do not consume more energy, the paper's central claim of default non-green behavior fails.

Watch

Extended reading notes

Core claim

The paper's central discovery is that the default behavior of these generative AI tools is non-green: left to their own devices, the models do not systematically emit the sustainable patterns that expert green-coding rules prescribe. Specifically, none of the three tools provided what the paper calls the most optimum solution for Rule 1, which asks for Java code to read a large file and count words, and Copilot's suggestions were non-green for at least one suggested solution on five of the six rules studied. ChatGPT failed to adhere on two of the six rules and BARD followed closely behind; the authors attribute the pattern to factors such as differences in training data and training methods. The paper presents this as an early profile of default behavior rather than as a ranking of one tool's superiority over another.

Load-bearing premise

The study's scoring assumes the chosen rule set plus the Java I/O energy study are a complete, transferable definition of green code, and that absence of the named pattern means non-green, without measuring energy or considering other equally efficient alternatives.

Editorial extensions

If this is right

Developers who accept default output from these tools will routinely embed non-green patterns in everyday tasks such as file I/O, string building, SQL schema creation, and Python file handling.
If the industry forecast cited by the paper holds, most professional developers will soon use AI-assisted coding, so default non-green behavior could propagate into a large share of newly written software.
Mitigation exists but is conditional: teams with model access can fine-tune on sustainable practices, while black-box users can append the relevant practice to the prompt, shifting some of the burden back to the developer.
Sustainability should become a standard dimension in evaluations of AI-generated code, alongside correctness, quality, and security, rather than an afterthought.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper's pattern-based scoring measures rule adherence, not kilowatt-hours; a metered replication could reveal that the actual energy penalty of a non-green default is small or large for these particular tasks.
Because the study runs each prompt once per conversational tool, the results capture a snapshot of model behavior; any updated model version could shift the default patterns and should be re-tested.
A cheap follow-up test is prompt augmentation: append 'use the most energy-efficient standard library' to the same prompts and see whether the green pattern appears, which would separate missing knowledge from missing elicitation.
The general conclusion that AI tools default to non-green code is directly supported only for the six rules, three languages, and three tools studied; extending it to other rule sets and languages is an inference, not a demonstrated result.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick read of the paper: it is a small exploratory study, but it does something the literature has not done yet—checks AI code-generation tools against known green coding rules. The methodology is straightforward and honest: pick six rules from CAST Green IT and one from a Java I/O energy study, craft two prompts per rule, run ChatGPT, BARD, and Copilot, and score outputs by whether the recommended pattern appears. That is a legitimate first cut, and the result—Copilot is non-green on five of six rules, the others miss several—is in line with what you would expect from tools trained on typical, not energy-optimized, code. The paper does not overclaim about which tool is better; it explicitly says the results are not representative.

The soft spots are real but they are the normal soft spots of an exploratory workshop paper. The biggest one is that greenness is scored by pattern presence, not by measured energy use. For Rule 1 the mapping is especially borrowed: the paper cites Rocha et al. for the claim that Scanner/FileInputStream are inefficient, but it does not show that the specific generated snippets differ measurably in energy from the recommended alternatives. So the strong phrase "default non-green behavior" is only as strong as the rule set's transferability. Also, only one conversational run per tool per prompt, no tool versions or dates, and no raw outputs in the paper—Table 1's color assignments are unverifiable without contacting the authors. The limitations section admits the rule set is small and not representative, which is good, but it does not flag the absence of raw data as a reproducibility gap.

None of this kills the paper. The central observation—that popular tools frequently ignore well-known sustainable patterns—is plausible and probably true. What the paper cannot support is the quantitative claim of "significant energy impact," which is asserted in the I/O discussion. For a four-page workshop paper, the evidence is about right.

Who is this for? Researchers in green software engineering and anyone thinking about AI-assisted coding's environmental footprint. It is a reasonable starting point for more rigorous studies, not a definitive measurement. I would send it to peer review—it deserves referee time as an early empirical claim, and the authors are clear about its limits. I would ask referees to push for raw data and version pinning in a revision.

For a reading group: yes, quick and discussable. I would cite it in related-work contexts with a caveat.

Referee Report

4 major / 5 minor

Summary. The paper reports an early, exploratory evaluation of whether three generative AI coding tools (GitHub Copilot, OpenAI ChatGPT, and Google BARD) produce code that follows sustainable coding practices. The authors select six rules: five from the CAST Green IT rule set and one constructed from a study of Java I/O API energy behavior (Rocha et al. [18]). For each rule they manually craft two natural-language prompts, obtain code outputs from the tools, and classify each output as green or non-green based on whether the recommended pattern (e.g., BufferedReader over Scanner, StringBuilder over string concatenation) is present. Table 1 reports that Copilot is non-green in at least one suggested solution for 5 of 6 rules, while ChatGPT and BARD also fail some rules. The paper concludes that the tools show a 'default non-green behavior' and suggests fine-tuning or prompt engineering as remedies.

Significance. If the central claim were supported, the paper would provide an important early warning for software sustainability: many developers now accept AI-generated code without scrutiny, and systematic non-adherence to known green coding rules would have a multiplicative energy impact. The study has some genuine strengths: it grounds the evaluation in an externally defined rule set rather than a self-fit metric, it uses manually crafted prompts that target realistic tasks, and it explicitly acknowledges its own limitations in Section 3. The evaluation is not circular in the sense of fitting parameters to tool outputs; its validity depends on whether the selected rules are a faithful proxy for energy efficiency. However, the empirical basis is too thin to establish 'default' behavior: there is one conversational run per tool-prompt pair, no repeated trials, no tool version or date pinning, no raw outputs, and no energy measurement. The paper is best read as a motivating exploratory study rather than a robust comparative evaluation.

major comments (4)

[Section 2.2 and Table 1] The central claim of 'default non-green behavior' rests on a single run per tool-prompt combination, but LLM outputs are stochastic and the settings used are not reported. The paper states that ChatGPT and BARD were accessed through their conversational UIs and Copilot through the VSCode extension, but it does not give model versions, dates of access, temperature settings, or the number of repeated attempts. Without repeated trials and version pinning, Table 1 describes one snapshot, not a 'default' behavior, so the abstract's generalization is not supported by the reported data.
[Section 2.1.1, Section 2.2, Table 1] Greenness is operationalized as the presence or absence of a rule-specific pattern, with no energy measurement of the generated snippets. For Rule 1, the paper cites Rocha et al. [18] to assert that Scanner and FileInputStream are not energy efficient, but it does not show that the energy difference between the tools' actual outputs and the recommended alternative is significant for the specific snippets, inputs, and contexts of this study. A solution lacking the preferred pattern is therefore not necessarily measurably less green in every realistic use, and a solution containing the pattern could still be inefficient in other ways. The phrase 'significant energy impact' in Section 2.2 is an extrapolation beyond the observed data.
[Section 2.1.3 and Table 1] The evaluation procedure is under-specified for ChatGPT and BARD. For Copilot, the paper gives a percentage formula (green suggestions divided by total suggestions), but for ChatGPT and BARD the table uses green/orange/red cells without defining how 'partial presence' is scored when only one answer is returned per prompt, how the two prompts per rule are aggregated, or how 'all presented solutions' is determined. Without this information, the reader cannot reproduce the classifications or verify counts such as Copilot's 5/10, 6/10, and 5/8 entries.
[Section 3 and Section 2.2] The paper itself acknowledges in Section 3 that the rule set is small and the results are not representative of one tool's superior capability, yet Section 2.2 draws a stronger comparative conclusion ('Copilot recommended solutions to be less sustainable than that of ChatGPT or BARD') and calls the results an 'irrefutable fact.' With only six rules, two prompts per rule, and one run per prompt, the data cannot support a cross-tool ranking or an 'irrefutable' statement without statistical testing and a larger sample.

minor comments (5)

[Section 2.1.3 and Table 1] The text says two prompts were created for each rule, but Table 1 presents two rows per rule; please clarify whether each row was evaluated separately and whether the reported color/percentage is per prompt or aggregated across both prompts.
[Section 2.2] Please state the exact access dates and, if possible, the underlying model versions or release identifiers for ChatGPT, BARD, and the Copilot extension; without this, the results cannot be replicated as models are updated frequently.
[Section 3] There is a minor typo: 'evaluation hundreds of LLMs' should be 'evaluating hundreds of LLMs.'
[References] Several references are web pages with access dates but no stable identifiers; adding archived links or DOI-style references would improve reproducibility.
[Figure 1] Figure 1 is an illustrative cartoon; it is not referenced in the text. Either cite it where the 'default' behavior is discussed or remove it to avoid confusion.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the evaluation applies external green-rule criteria to fresh tool outputs, with no fitted parameter or self-citation chain driving the result.

full rationale

The paper's evaluation chain is: (i) select green rules from CAST Green IT and from the external Java I/O energy study by Rocha et al.; (ii) craft natural-language prompts tied to those rules; (iii) run ChatGPT, BARD, and Copilot on the prompts; (iv) classify each output by whether it contains the recommended pattern. None of these steps fits a parameter to tool outputs and then 'predicts' the same data. The central claim that tools exhibit default non-green behavior is a direct application of the external rule set: a solution is marked non-green when it lacks the stated pattern (e.g., BufferedReader vs. Scanner), which is the operational definition of rule adherence, not a circular derivation. The paper does not cite its own prior work for the energy claims; the only self-reference ([13], on developer awareness) is not load-bearing. Concerns about whether rule adherence actually implies measurable energy savings are validity/correctness concerns, not circularity. The limitation section explicitly disclaims representativeness, further confirming the claims are bounded rather than derived from themselves.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The central claim depends on accepting the selected green rules as valid energy proxies and on the authors' manual scoring; no numerical fitting is involved.

assumptions (5)

domain assumption CAST Green IT rules and the Java I/O API energy study are a valid ground truth for greenness.
Invoked in Section 2.1.1 to justify the five CAST rules plus one I/O rule; outputs are scored green or non-green solely by presence of these patterns.
domain assumption A single generated response per tool and prompt is representative of the tool's default behavior.
Used in Section 2.2 with no repeated runs or variance reporting; conversational LLMs are stochastic and UI versions change.
domain assumption Manual inspection by the authors is reliable and consistent.
Used in Section 2.2 to classify each generated solution; no inter-rater reliability or detailed rubric is provided.
domain assumption The NL prompts trigger the rule's intended coding context without biasing the tool toward the green pattern.
Section 2.1.3 states prompts are rule-directed by design but do not mention sustainability; this is necessary to interpret outputs as default behavior.
domain assumption For each prompt there is a uniquely 'most optimum' green solution represented by the rule.
Section 2.2 says for Rule 1 none gave the most optimum solution; the paper assumes the recommended API or pattern is the best option, without measuring alternatives.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Do Generative AI Tools Ensure Green Code? An Investigative Study." pith.science (2026). https://pith.science/paper/V7BZN7D2

@misc{pith2026250608790,
  author       = {Pith},
  title        = {Pith review of: Do Generative AI Tools Ensure Green Code? An Investigative Study},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/V7BZN7D2}},
  note         = {Machine review of arXiv:2506.08790}
}

read the original abstract

Software sustainability is emerging as a primary concern, aiming to optimize resource utilization, minimize environmental impact, and promote a greener, more resilient digital ecosystem. The sustainability or "greenness" of software is typically determined by the adoption of sustainable coding practices. With a maturing ecosystem around generative AI, many software developers now rely on these tools to generate code using natural language prompts. Despite their potential advantages, there is a significant lack of studies on the sustainability aspects of AI-generated code. Specifically, how environmentally friendly is the AI-generated code based upon its adoption of sustainable coding practices? In this paper, we present the results of an early investigation into the sustainability aspects of AI-generated code across three popular generative AI tools - ChatGPT, BARD, and Copilot. The results highlight the default non-green behavior of tools for generating code, across multiple rules and scenarios. It underscores the need for further in-depth investigations and effective remediation strategies.

Figures

Figures reproduced from arXiv: 2506.08790 by the authors.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

26 extracted references · 21 canonical work pages

[18]

Gilson Rocha, Fernando Castor, and Gustavo Pinto. 2019. Comprehending energy behaviors of java i/o apis. In2019 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM) . IEEE, 1–12

work page 2019
[1]

Google Blog. 2023. An important next step on our AI journey. Retrieved No- vember 12, 2023 from https://blog.google/technology/ai/bard-google-ai-search- updates/

work page 2023
[2]

Déaglán Connolly Bree and Mel Ó Cinnéide. 2020. Inheritance versus delegation: which is more energy efficient?. InProceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops. 323–329. 1Stanford HELM : https://crfm.stanford.edu/helm/latest/#/models

work page 2020
[3]

Xueying Du, Mingwei Liu, Kaixin Wang, Hanlin Wang, Junwei Liu, Yixuan Chen, Jiayi Feng, Chaofeng Sha, Xin Peng, and Yiling Lou. 2023. ClassEval: A Manually- Crafted Benchmark for Evaluating LLMs on Class-level Code Generation. ArXiv abs/2308.01861 (2023)

arXiv 2023
[4]

Blair, and Adrian Friday

Charlotte Freitag, Mike Berners-Lee, Kelly Widdicks, Bran Knowles, Gordon S. Blair, and Adrian Friday. 2021. The real climate and transformative impact of ICT: A critique of estimates, trends, and regulations. Patterns 2, 9 (2021), 100340

work page 2021
[5]

Accessed - 02/12/2023

gartner.com. Accessed - 02/12/2023. Set Up Now for AI to Augment Soft- ware Development. https://www.gartner.com/en/articles/set-up-now-for-ai- to-augment-software-development

work page 2023
[6]

Github. 2023. The economic impact of the AI-powered developer lifecy- cle and lessons from GitHub Copilot. Retrieved November 12, 2023 from https://github.blog/2023-06-27-the-economic-impact-of-the-ai-powered- developer-lifecycle-and-lessons-from-github-copilot/

work page 2023
[7]

Accessed - 02/12/2023

github.com. Accessed - 02/12/2023. Github Copilot. https://github.com/features/ copilot

work page 2023

Show all 26 references

[8]

Accessed - 02/12/2023

github.com. Accessed - 02/12/2023. Survey reveals AI’s impact on the developer experience

2023
[9]

Accessed - 02/12/2023

google.com. Accessed - 02/12/2023. BARD. https://bard.google.com

2023
[10]

Mourão, and Ivan Machado

Leila Karita, Brunna C. Mourão, and Ivan Machado. 2019. Software Industry Awareness on Green and Sustainable Software Engineering: A State-of-the- Practice Survey. In Proceedings of the XXXIII Brazilian Symposium on Software Engineering (Salvador, Brazil) (SBES ’19)

2019
[11]

Avila, Jacob Brunelle, and Baba Mamadou Camara

Raphaël Khoury, Anderson R. Avila, Jacob Brunelle, and Baba Mamadou Camara
[12]

J. T. Liang, C. Yang, and B. A. Myers. 2024. A Large-Scale Survey on the Usability of AI Programming Assistants: Successes and Challenges. In 2024 IEEE/ACM 46th International Conference on Software Engineering (ICSE) . 605–617

2024
[13]

Rohit Mehra, Vibhu Saujanya Sharma, Vikrant Kaulgud, Sanjay Podder, and Adam P. Burden. 2022. Towards a Green Quotient for Software Projects. In 2022 IEEE/ACM 44th International Conference on Software Engineering: Software Engineering in Practice (ICSE-SEIP) . 295–296

2022
[14]

Accessed - 02/12/2023

meta.com. Accessed - 02/12/2023. Code Llama. https://ai.meta.com/llama/

2023
[15]

Accessed - 02/12/2023

openai.com. Accessed - 02/12/2023. ChatGPT. https://chat.openai.com

2023
[16]

Gustavo Pinto and Fernando Castor. 2017. Energy Efficiency: A New Concern for Application Software Developers. Commun. ACM 60, 12 (nov 2017), 68–75

2017
[17]

Reuters. 2023. ChatGPT sets record for fastest-growing user base. Retrieved No- vember 12, 2023 from https://www.reuters.com/technology/chatgpt-sets-record- fastest-growing-user-base-analyst-note-2023-02-01/

2023
[19]

Cagri Sahin, Furkan Cayci, Irene Lizeth Manotas Gutiérrez, James Clause, Fouad Kiamilev, Lori Pollock, and Kristina Winbladh. 2012. Initial explorations on design pattern energy usage. In 2012 First International Workshop on Green and Sustainable Software (GREENS). 55–61

2012
[20]

Accessed - 02/12/2023

tabnine.com. Accessed - 02/12/2023. Tabnine. https://www.tabnine.com

2023
[21]

Accessed - 02/12/2023

CAST Technologies. Accessed - 02/12/2023. CAST Rules Documentation. https: //technologies.castsoftware.com/rules?sec=idx_green&ref=||

2023
[22]

Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971 (2023)

2023 arXiv
[23]

Ac- cessed - 02/12/2023

United-Nations-Environment-Programme-Copenhagen-Climate-Centre. Ac- cessed - 02/12/2023. Greenhouse gas emissions in the ICT sector: trends and methodologies

2023
[24]

Glassman

Priyan Vaithilingam, Tianyi Zhang, and Elena L. Glassman. 2022. Expectation vs Experience - Evaluating the Usability of Code Generation Tools Powered by Large Language Models. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (CHI EA ’22) ....

2022
[25]

Burak Yetistiren, Isik Özsoy, Miray Ayerdem, and Eray Tüzün. 2023. Evaluating the Code Quality of AI-Assisted Code Generation Tools: An Empirical Study on GitHub Copilot, Amazon CodeWhisperer, and ChatGPT. ArXiv abs/2304.10778 (2023)

2023 arXiv
[2023]

How Secure is Code Generated by ChatGPT? arXiv:2304.09655 [cs.CR]

arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [18]

Gilson Rocha, Fernando Castor, and Gustavo Pinto. 2019. Comprehending energy behaviors of java i/o apis. In2019 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM) . IEEE, 1–12

work page 2019

[2] [1]

Google Blog. 2023. An important next step on our AI journey. Retrieved No- vember 12, 2023 from https://blog.google/technology/ai/bard-google-ai-search- updates/

work page 2023

[3] [2]

Déaglán Connolly Bree and Mel Ó Cinnéide. 2020. Inheritance versus delegation: which is more energy efficient?. InProceedings of the IEEE/ACM 42nd International Conference on Software Engineering Workshops. 323–329. 1Stanford HELM : https://crfm.stanford.edu/helm/latest/#/models

work page 2020

[4] [3]

Xueying Du, Mingwei Liu, Kaixin Wang, Hanlin Wang, Junwei Liu, Yixuan Chen, Jiayi Feng, Chaofeng Sha, Xin Peng, and Yiling Lou. 2023. ClassEval: A Manually- Crafted Benchmark for Evaluating LLMs on Class-level Code Generation. ArXiv abs/2308.01861 (2023)

arXiv 2023

[5] [4]

Blair, and Adrian Friday

Charlotte Freitag, Mike Berners-Lee, Kelly Widdicks, Bran Knowles, Gordon S. Blair, and Adrian Friday. 2021. The real climate and transformative impact of ICT: A critique of estimates, trends, and regulations. Patterns 2, 9 (2021), 100340

work page 2021

[6] [5]

Accessed - 02/12/2023

gartner.com. Accessed - 02/12/2023. Set Up Now for AI to Augment Soft- ware Development. https://www.gartner.com/en/articles/set-up-now-for-ai- to-augment-software-development

work page 2023

[7] [6]

Github. 2023. The economic impact of the AI-powered developer lifecy- cle and lessons from GitHub Copilot. Retrieved November 12, 2023 from https://github.blog/2023-06-27-the-economic-impact-of-the-ai-powered- developer-lifecycle-and-lessons-from-github-copilot/

work page 2023

[8] [7]

Accessed - 02/12/2023

github.com. Accessed - 02/12/2023. Github Copilot. https://github.com/features/ copilot

work page 2023