Pith. sign in

REVIEW 4 major objections 4 minor 2 cited by

Evaluating the Energy-Efficiency of the Code Generated by LLMs

T0 review · 4 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that LLM-generated code, even when functionally correct, is less energy-efficient than canonical human-written solutions across 878 coding problems: the best models are 17–21% behind on average, the worst more than twice…

desk verdict Solid benchmark data on LLM code energy efficiency, but the headline claim overstates a baseline that even the paper's own case study contradicts. read the letter →

arxiv 2505.20324 v1 pith:3E7B2UWW submitted 2025-05-23 cs.SE cs.AI

Md Arman Islam , Devi Varaprasad Jonnala , Ritika Rekhi , Pratik Pokharel , Siddharth Cilamkoti , Asif Imran , Tevfik Kosar , Bekir Turkkan This is my paper · ORCID
classification cs.SEcs.AI
keywords energyefficiencyLLMcodegenerationgreensoftwaresustainableAIoptimizationmeasurementbenchmarkingruntimeanalysis
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that code produced by large language models, although often functionally correct, is on average less energy-efficient than canonical human-written solutions for the same programming problems. The authors measured processor-package and RAM energy for solutions generated by 20 LLMs across 878 interview-style coding problems, using human-written canonical solutions as the baseline. They report that human solutions are about 1.17 times more energy-efficient than DeepSeek-v3, 1.21 times more efficient than GPT-4o, and over 2 times more efficient than Grok-2 and Gemini-1.5-Pro. If this is right, current correctness-focused evaluations of AI code generators miss a real environmental and economic cost, and model choice plus algorithmic optimization become first-order concerns for sustainable software development.

What carries the argument

The central mechanism is a relative-efficiency comparison: each LLM solution's total energy (processor package plus RAM, with measured idle power subtracted) and runtime are divided by the same metrics for the problem's canonical human solution, producing a ratio in which values above 1 mean the LLM used more energy. This ratio, averaged over two common-problem sets derived after filtering out broken problems, is what carries the conclusion that human solutions are more energy-efficient.

What would settle it

Gather alternative human-written solutions for a meaningful sample of the 878 problems and measure whether a substantial fraction beat the canonical solutions on energy; if they do, the reported human-vs-LLM ratios are an artifact of baseline choice rather than a general human advantage.

Watch

Extended reading notes

Core claim

The paper's central claim is that LLM-generated code is measurably less energy-efficient than canonical human-written solutions once correctness is held fixed. On the 298 problems that all 20 LLMs solved correctly, canonical solutions averaged 5.77 J of total energy while the most efficient LLM, DeepSeek-v3, averaged 5.91 J, and the least efficient, GPT-4 Turbo, averaged 12.00 J. On the broader set of 576 problems solved by 11 LLMs, the gap widens: canonical solutions averaged 5.46 J whereas DeepSeek-v3 averaged 6.37 J. The paper also identifies algorithmic categories where the gap is extreme: for dynamic programming, backtracking, and bit manipulation, some LLM solutions consumed up to 450 times more energy than the canonical solution, often because they omit pruning, duplicate logic, or choose worse algorithms.

Load-bearing premise

The load-bearing assumption is that the canonical human solutions used as baselines are genuinely near-optimal in energy for each problem, and that each problem's 100 test cases capture correctness fully; the paper itself removed 122 problems whose canonical solutions or tests were broken, so the remaining baselines carry the entire comparison.

Editorial extensions

If this is right

  • Energy efficiency should become a reported metric in LLM code-generation benchmarks, alongside functional correctness.
  • Model selection has a direct energy consequence: switching from Gemini-1.5-Pro or Grok-2 to DeepSeek-v3 roughly halves the execution energy of generated code.
  • Optimization pressure on dynamic programming, backtracking, and bit manipulation problems would yield the largest sustainability gains.
  • The models that are cheapest to query are not the ones whose output is cheapest to run, so cost analyses that ignore execution energy can mislead.
  • Even after allowing up to 25 regeneration attempts to obtain correct code, no LLM matched the average energy of human canonical solutions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the dominant cause of the gap is missing algorithmic optimizations like pruning, then energy-aware repair loops that feed measured energy back into the model could close much of the gap; the paper does not test this.
  • The measured energy covers only execution of the generated code, not the energy spent by the LLM during generation; including generation-side energy would make the total environmental cost of LLM-generated code larger, not smaller.
  • Because the results are for Python only, the ranking may differ in compiled languages where constant-factor overheads that inflate LLM energy use could shrink.
  • Since the benchmark problems are publicly available and may have appeared in model training data, the gap could shift on problems created after the models' training cutoffs.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper benchmarks the energy efficiency of code generated by 20 LLMs on 878 LeetCode problems drawn from EffiBench, comparing the LLM outputs against EffiBench's canonical human-written solutions. The authors measure package and RAM energy with perf, runtime, and memory, and also report pass rates, token counts, and API costs. Their headline claim is that human canonical solutions are on average about 1.17x more energy-efficient than DeepSeek-v3, 1.21x more than GPT-4o, and over 2x more than Grok-2 and Gemini-1.5-Pro, with much larger gaps for dynamic programming, backtracking, and bit-manipulation problems. The paper also identifies DeepSeek-v3 and GPT-4o as the most energy-efficient LLMs and provides two case studies explaining why some LLM solutions consume far more energy than the canonical baseline.

Significance. If the central result holds, this is a useful and timely empirical contribution to green software engineering and LLM code generation. The study covers a broad set of models (20) and problems (878), uses a public benchmark, and provides a reproducible workflow (the anonymous repository is a concrete asset). The energy measurement methodology, including idle-power baseline subtraction, cooldown periods, and five repeated runs, is more careful than many prior token-based efficiency studies. The breakdown by difficulty and algorithmic category is valuable for practitioners. However, the significance is currently limited by an unreliable baseline claim: the paper's own case study shows that a canonical 'most efficient human' solution can be 5x less efficient than the best LLM solution. The headline ratios are also drawn from a 576-problem/11-model subset rather than the full 878-problem/20-model set, and several aggregate statements are contradicted by the paper's own per-category tables. These issues need to be fixed before the quantitative conclusions can be taken as a general human-versus-LLM efficiency gap.

major comments (4)
  1. [Appendix F.2, §3] The paper's own case study contradicts the assumption that EffiBench canonical solutions are the 'most efficient' human-written solutions. For LeetCode Problem 740 (Delete and Earn), the canonical solution uses an O(max(nums)) list-based algorithm and consumes 5x more energy than the best LLM solutions, which use O(len(nums)) iterative DP. Since §3 states that EffiBench pairs problems with 'the most efficient solutions from the LeetCode discussion forum,' this counterexample shows the baseline is not a reliable lower-energy reference. The authors should report the per-problem distribution of the relative energy ratio (Eq. 13) and the fraction of problems where each LLM beats the canonical solution, rather than only aggregate ratios in Tables 8 and 9. Without this, the headline conclusion that human-written solutions are more energy-efficient than LLM-generated code may be an artifact of the particular canonical solutions chosen.
  2. [Abstract, §4.2.2, Tables 7 and 9] The abstract and introduction present the 1.17x, 1.21x, and over-2x ratios as if they apply to the full 878-problem, 20-model study, but these numbers come from Benchmark Set-II, which contains only 576 problems successfully solved by 11 LLMs. The all-20-model comparison in Benchmark Set-I is restricted to 298 common problems, with only 30 hard problems (Table 4). The authors should state these subset sizes explicitly in the abstract and introduction, and qualify the generalization of the headline ratios accordingly.
  3. [§4.2.1, §5, Tables 13, 16, 25, 27] The text overclaims that canonical solutions 'consistently outperform all evaluated LLMs' and that LLM energy efficiency is 'always poorer,' but the paper's own tables contain numerous counterexamples. For example, in Table 13 (Benchmark Set-II, easy problems), Claude 3.5 Haiku has average total energy 5.1176 J versus canonical 5.1220 J; in Table 16 (Greedy, Benchmark Set-I), at least nine LLM solutions have lower average total energy than canonical; in Table 25 (Bit Manipulation, Benchmark Set-I), GPT-4o, DeepSeek-v3, and several others are below canonical; and in Table 27 (Greedy, Benchmark Set-II), Grok 2 and DeepSeek-v3 are below canonical. The central claim should be reframed as 'on average across the benchmark' and 'for certain categories,' not as a universal property. The authors should also provide confidence intervals or significance tests for the aggregate differences, since some reported gaps (e.g., 5.77 J vs 5.91 J) may be within measurement noise.
  4. [Abstract, Introduction, Appendix F.1] The statement that LLM-generated code can consume 'up to 450 times more energy' for dynamic programming, backtracking, and bit manipulation is based on a single problem (LeetCode 2305, Fair Distribution of Cookies) discussed in Appendix F.1. The abstract and introduction present this as a general pattern for entire algorithmic groups. The authors should either provide the per-problem maximum across all problems in each category or explicitly label the 450x figure as a single-case example. A distribution of per-problem relative energies is needed to support any claim about the maximum or about category-level behavior.
minor comments (4)
  1. [§4.1, Table 3] The text says GPT-4o has a Pass@10 score of '89.1%', but Table 3 reports 89.7%; the values should be reconciled.
  2. [§4.2.2] The paper refers to '11 LLMs' in Benchmark Set-II but never lists which 11 models are included; the reader must infer the set from Table 7. Please state the model list explicitly.
  3. [Appendix F.2] Bullet 4 of the Problem 740 case study says the worst solution is produced by 'Claude-Sonnet and LLaMA 3.1 70B,' but bullet 1 lists LLaMA 3.1 70B among the best solutions. This appears to be a typo (likely LLaMA 3.3 70B) and should be corrected.
  4. [§3.1, Table 1] The table caption says 'TOTAL' but one problem can belong to multiple algorithmic categories, so the TOTAL column is not the sum of the category columns; this should be clarified in the caption or text.

Circularity Check

0 steps flagged · score 0.0 of 10

External measurement against EffiBench baselines; no fitted parameters or self-citation chain, so no circularity.

full rationale

The paper's derivation chain is an empirical measurement: it takes EffiBench's canonical human solutions as an external baseline, runs the same 100 test cases, measures package/RAM energy with perf, subtracts idle power, averages over five runs, and forms ratios via Eq. (13) (Relative_m,ri = V_m,ri / V_canonical,ri). No parameter is fitted to the outcome, no equation defines the conclusion into the inputs, and the energy ratios are directly falsifiable and are reported per benchmark set and per algorithmic category. The only premise that could look circular is the EffiBench label "most efficient solutions ... canonical human-written solutions" (Section 3), but the paper does not rely on that label for the measurement; it measures energy directly, and the paper's own Appendix F.2 (LeetCode 740) shows the canonical solution consuming 5x more energy than the best LLM solution, which weakens the human-superiority generalization but is an empirical counterexample rather than reasoning in a circle. There are no load-bearing self-citations and no imported uniqueness theorems; citations such as EffiBench are external benchmark definitions, not author-overlapping justifications. Given the rubric, the correct finding is no significant circularity.

Assumptions & free parameters 0 free parameters · 7 assumptions · 0 invented entities

The empirical claims depend on the provenance and quality of the EffiBench baseline, the validity of perf RAPL measurements, and the representativeness of the common-problem subsets and single-node hardware. No free parameters are fitted; temperature settings are model defaults, and thresholds such as the 30-second baseline, 10-second cooldown, and 5 runs are experimental conventions rather than fitted constants.

assumptions (7)
  • domain assumption EffiBench canonical solutions are the most efficient human-written solutions for each problem.
    All human-vs-LLM energy ratios compare against this baseline, so if the canonical choice is not near-optimal the central comparison loses meaning. Invoked in Section 3 when the benchmark is introduced.
  • domain assumption The 100 EffiBench test cases per problem are complete and correct.
    Correctness is judged by passing these tests, and 122 problems were discarded because tests or canonical solutions were broken. Remaining results inherit the test suite's quality. Sections 3.1 and 3.3.
  • domain assumption perf RAPL energy counters with idle-power subtraction accurately measure code energy.
    Package and RAM energy are computed with perf and Equation 2; any counter or baseline error shifts every absolute and relative value. Section 3.4.
  • domain assumption The common-problem subsets are representative of the full 878-problem set.
    Energy comparisons only use 298 or 576 problems solved by all or 11 models; the difficulty and algorithm distribution changes, so averages may not generalize to the full dataset. Section 4.2.
  • domain assumption Five repeated runs, averaged, provide adequate statistical precision.
    No variance or confidence intervals are supplied, so readers cannot assess whether model differences are significant. Section 3.4 and all result tables.
  • domain assumption A single Intel Xeon Gold 6126 node is representative of realistic deployment hardware.
    All energy, runtime, and memory numbers come from one Chameleon Cloud instance; energies are expected to vary with processor and memory technology. Section 3.6.
  • domain assumption LeetCode interview problems are a valid proxy for real-world coding workloads.
    The benchmark's external validity rests on this analogy, stated in Sections 1 and 3.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Evaluating the Energy-Efficiency of the Code Generated by LLMs." pith.science (2026). https://pith.science/paper/3E7B2UWW

@misc{pith2026250520324,
  author       = {Pith},
  title        = {Pith review of: Evaluating the Energy-Efficiency of the Code Generated by LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3E7B2UWW}},
  note         = {Machine review of arXiv:2505.20324}
}
read the original abstract

As the quality of code generated by Large Language Models (LLMs) improves, their adoption in the software industry for automated code generation continues to grow. Researchers primarily focus on enhancing the functional correctness of the generated code while commonly overlooking its energy efficiency and environmental impact. This paper investigates the energy efficiency of the code generated by 20 popular LLMs for 878 programming problems of varying difficulty levels and diverse algorithmic categories selected from the LeetCode platform by comparing them against canonical human-written solutions. Although LLMs can produce functionally correct results in most cases, our findings show that the performance and energy efficiency of LLM-produced solutions are often far below those of human-written solutions. Among the studied LLMs, DeepSeek-v3 and GPT-4o generate the most energy-efficient code, whereas Grok-2 and Gemini-1.5-Pro are among the least energy-efficient models. On average, human-generated canonical solutions are approximately 1.17 times more energy efficient than DeepSeek-v3, 1.21 times more energy efficient than GPT-4o, and over 2 times more energy efficient than Grok-2 and Gemini-1.5-Pro. For specific algorithmic groups such as dynamic programming, backtracking, and bit manipulation, LLM-generated code can consume up to 450 times more energy than human-generated canonical solutions.

Figures

Figures reproduced from arXiv: 2505.20324 by the authors.

Figure 1
Figure 1. End-to-end experimentation workflow. 12 [PITH_FULL_IMAGE:figures/full_fig_p012_1.png] view at source ↗
Figure 2
Figure 2. Prompt structure used for code generation. [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗
Figure 3
Figure 3. The methodology followed for code generation. [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Flowchart showing the experimental process. [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]
Figure 5
Figure 5. Figure 5: The above figure shows the canonical solution and the code generated by different LLMs [PITH_FULL_IMAGE:figures/full_fig_p026_5.png]
Figure 6
Figure 6. Figure 6: The above figure shows the canonical solution and the code generated by different LLMs [PITH_FULL_IMAGE:figures/full_fig_p027_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Rethinking Code Performance Benchmarks for LLMs

    cs.SE 2026-07 conditional novelty 6.0 of 10

    Re-evaluating four LLM code-efficiency benchmarks with 30-run statistical testing shows 93.89% of 'performant' implementations are indistinguishable from baselines; a multi-agent test-generation framework reveals hidd...

  2. Energy-Aware Code Generation with LLMs: Benchmarking Small vs. Large Language Models for Sustainable AI Programming

    cs.SE 2025-08 reject novelty 4.0 of 10

    On 150 LeetCode problems, GPT-4.0 and DeepSeek-Reasoner beat three 3B-parameter models on correctness and speed; the 52% energy-efficiency claim counts any of three SLMs on correct outputs, not a per-model advantage.

Reference graph

Works this paper leans on

42 extracted references · 34 canonical work pages · cited by 2 Pith papers

  1. [1]

    Learn to code sustainably: An empirical study on llm-based green code generation

    Tina Vartziotis, Ippolyti Dellatolas, George Dasoulas, Maximilian Schmidt, Florian Schneider, Tim Hoffmann, Sotirios Kotsopoulos, and Michael Keckeisen. Learn to code sustainably: An empirical study on llm-based green code generation. arXiv preprint arXiv:2403.03344, 2024

    arXiv 2024

  2. [2]

    Electricity consumption by ict: Facts, trends, and measurements

    Erol Gelenbe. Electricity consumption by ict: Facts, trends, and measurements. Ubiquity, 2023(August), August 2023

    work page 2023

  3. [3]

    Ai’s growing carbon footprint, June 2023

    State of the Planet. Ai’s growing carbon footprint, June 2023. Accessed: 2025-04-21

    work page 2023

  4. [4]

    Facts & figures, 2023

    Air Transport Action Group. Facts & figures, 2023. Accessed: 2025-04-21

    work page 2023

  5. [5]

    Carbon emissions and large neural network training

    David Patterson, Joseph Gonzalez, Quoc Le, Chen Liang, Lora Munguia, Dieter Rothchild, Denny So, Maud Texier, and Jeff Dean. Carbon emissions and large neural network training. arXiv preprint arXiv:2104.10350, 2021

    arXiv 2021

  6. [6]

    Assessing ict global emissions footprint: Trends to 2040 & recommen- dations

    Lotfi Belkhir and Ahmed Elmeligi. Assessing ict global emissions footprint: Trends to 2040 & recommen- dations. Journal of cleaner production, 177:448–463, 2018

    work page 2018

  7. [7]

    Green ai: Energy-efficient training and inference of deep neural networks

    Kai Li, Guojian Wu, Chao Wang, Forest Agostinelli, and Kurt Keutzer. Green ai: Energy-efficient training and inference of deep neural networks. Journal of Machine Learning Research, 24(163):1–43, 2023

    work page 2023

  8. [8]

    Effibench: Benchmarking the efficiency of automatically generated code

    Dong Huang, Yuhao Qing, Weiyi Shang, Heming Cui, and Jie M Zhang. Effibench: Benchmarking the efficiency of automatically generated code. In Proceedings of the 38th Conference on Neural Information Processing Systems (NeurIPS 2024), 2024

    work page 2024

Show all 42 references
  1. [9]

    Green software engineering: Metrics, practices, and tools

    Birgit Penzenstadler, Ankita Raturi, Debra Richardson, and Bill Tomlinson. Green software engineering: Metrics, practices, and tools. IEEE Software, 31(3):22–27, 2014

    2014

  2. [10]

    Green software engineering: The science of sustainable software development

    Roberto Verdecchia, Patricia Lago, and Christof Ebert. Green software engineering: The science of sustainable software development. IEEE Software, 39(6):12–17, 2022

    2022

  3. [11]

    Green measurement metrics towards a sustainable software: A systematic literature review

    Faisal Ahmed et al. Green measurement metrics towards a sustainable software: A systematic literature review. IEEE Conference Publication, 2017

    2017

  4. [12]

    Gallagher

    Xueliang Li and John P. Gallagher. An energy-aware programming approach for mobile application development guided by a fine-grained energy model. CoRR, abs/1605.05234, 2016

    2016 arXiv

  5. [13]

    Energy efficiency in sustainable software development: Clean code approaches

    Anayo Chukwu Ikegwu. Energy efficiency in sustainable software development: Clean code approaches. ResearchGate, 2024

    2024

  6. [14]

    Advancing green comput- ing: Practices, strategies, and impact in modern software development for environmental sustainability

    Mamoon Rashid, Adil Hussain, Syed Qasim Shah, and Zeeshan Ahmad. Advancing green comput- ing: Practices, strategies, and impact in modern software development for environmental sustainability. ResearchGate, 2024

    2024

  7. [15]

    Sustainable software engineering: Green coding’s impact on climate change

    Devsu. Sustainable software engineering: Green coding’s impact on climate change. Devsu Blog, 2024

    2024

  8. [16]

    Towards the systematic reporting of the energy and carbon footprints of machine learning

    Peter Henderson, Jieru Hu, Joshua Romoff, Emma Brunskill, Dan Jurafsky, and Joelle Pineau. Towards the systematic reporting of the energy and carbon footprints of machine learning. Journal of Machine Learning Research, 21(248):1–43, 2020

    2020

  9. [17]

    A holistic approach to environmentally sustainable computing

    Jay Patil, Abhijeet Deshpande, Aldo Badano, and Rahul Sharma. A holistic approach to environmentally sustainable computing. Innovations in Systems and Software Engineering , 2023

    2023

  10. [18]

    Green algorithms: Quantifying the carbon footprint of computation

    Loïc Lannelongue, Jason Grealey, and Michael Inouye. Green algorithms: Quantifying the carbon footprint of computation. 2020

    2020

  11. [19]

    Enamel: Efficency automatic evaluator

    Wei Shi, Yuxuan Li, and Jielun Zhang. Enamel: Efficency automatic evaluator. In OpenReview, 2024

    2024

  12. [20]

    Neural code intelligence: A new era in software engineering

    Ziyang Chen, Veselin Hellendoorn, Lambdax, and Premkumar Devanbu. Neural code intelligence: A new era in software engineering. Communications of the ACM, 66(7):84–94, 2023. 10

    2023

  13. [21]

    Functional correctness assessment of ai-generated code: Challenges and opportunities

    Luis Cruz, Rui Abreu, and David Lo. Functional correctness assessment of ai-generated code: Challenges and opportunities. IEEE Transactions on Software Engineering, 49(8):3401–3418, 2023

    2023

  14. [22]

    On evaluating the efficiency of source code generated by llms

    Changan Niu, Ting Zhang, Chuanyi Li, Bin Luo, and Vincent Ng. On evaluating the efficiency of source code generated by llms. arXiv preprint arXiv:2404.06041, 2024

    2024 arXiv

  15. [23]

    Unveiling inefficiencies in llm-generated code: Toward a comprehensive taxonomy

    Konstantinos Koutsouris et al. Unveiling inefficiencies in llm-generated code: Toward a comprehensive taxonomy. arXiv preprint arXiv:2503.06327, 2024

    2024 arXiv

  16. [24]

    Energy-aware prompt engineering for efficient code generation

    Yuxuan Lu, Chuan Yang, Shiyu He, and Sheng Jiang. Energy-aware prompt engineering for efficient code generation. In Proceedings of the 45th International Conference on Software Engineering , pages 1265–1276, 2023

    2023

  17. [25]

    Towards energy-efficient code optimization with large language models

    Muhammad Bilal et al. Towards energy-efficient code optimization with large language models. arXiv preprint arXiv:2410.09241, 2024

    2024 arXiv

  18. [26]

    Green-code: Optimizing energy efficiency in large language models for code generation

    Shashikant Ilager, Luca Bernardi, and Fanggang Liang. Green-code: Optimizing energy efficiency in large language models for code generation. arXiv preprint arXiv:2501.11006, 2025

    2025 arXiv

  19. [27]

    Reinforcement learning with energy consumption feedback for efficient code generation

    Lei Wang, Anran Liu, Sheng Shi, Yu Meng, and Qiang Yang. Reinforcement learning with energy consumption feedback for efficient code generation. In Proceedings of the ACM Conference on Sustainable Computing, pages 123–134, 2024

    2024

  20. [28]

    Generating energy-efficient code with llms

    Benjamin Cappendijk, Isaac Meijer, Frederico Pratas, and Patricia Lago. Generating energy-efficient code with llms. arXiv preprint arXiv:2411.10599, 2024

    2024 arXiv

  21. [29]

    Problems, 2025

    LeetCode. Problems, 2025. Accessed: 2025-04-21

    2025

  22. [30]

    perf-stat— Linux manual page , 2023

    Linux manual page. perf-stat— Linux manual page , 2023

    2023

  23. [31]

    Gunawi, Cody Hammock, Joe Mambretti, Alexander Barnes, François Halbach, Alex Rocha, and Joe Stubbs

    Kate Keahey, Jason Anderson, Zhuo Zhen, Pierre Riteau, Paul Ruth, Dan Stanzione, Mert Cevik, Jacob Colleran, Haryadi S. Gunawi, Cody Hammock, Joe Mambretti, Alexander Barnes, François Halbach, Alex Rocha, and Joe Stubbs. Lessons learned from the chameleon testbed. In Proceedin...

    2020

  24. [32]

    The best solution is the solution generated by LLMs Claude Haiku and GPT-4 Turboas they apply effective pruning strategies to eliminate redundant calculations

  25. [33]

    The Canonical Solution has a slightly higher energy consumption than the best solution as it misses one pruning strategy of eliminating redundant calculation of symmetric paths

  26. [34]

    GPT-4o and Gemini 2.0 Flash Lite generate solutions that closely resemble the canonical version, and therefore demonstrate similar energy consumption profiles

  27. [35]

    However, it does apply the pruning strategy to remove redundant calculation of symmetric paths

    Claude 3.5 Sonnet produces a solution that has 15 times higher energy consumption as compared to the best solution as it uses a minimum unfairness value that resets with each recursive call and lacks 25 other checks that can reduce redundant computations. However, it does appl...

  28. [36]

    DeepSeek v3 produces a slightly worse solution than the canonical solution. This solution uses a minimum unfairness value that resets with each recursive call and lacks the symmetric path check, leading to more unnecessary computations, and thus resulting in energy consumption...

  29. [37]

    Although the logic produced by both LLMs is very similar, LLaMA 3.3 70B does more redundant calculations by evaluating the branches with the same values

    LLaMA 3.3 70B produces an even worse solution than DeepSeek v3. Although the logic produced by both LLMs is very similar, LLaMA 3.3 70B does more redundant calculations by evaluating the branches with the same values. This results in energy consumption which is 3 times more th...

  30. [38]

    Gemini 1.5 Pro and Gemini 2.0 Flash generate the worst solutions. Both LLMs generate the same solution, which does not have any pruning logic in place, resulting in up to 450 times the energy consumption of the canonical solution and 548 times the energy consumption of the bes...

  31. [39]

    These models use an optimal approach using a few variables to iteratively compute the result

    The best solution is generated by GPT-4o, Grok2, LLaMA 3.1 70B, and Gemini 2.0 Flash-Lite. These models use an optimal approach using a few variables to iteratively compute the result. Their solutions achieve a time complexity of O(len(nums)), resulting in the lowest energy co...

  32. [40]

    This implementation relies on constructing a list with a size proportional to the maximum value in the input array, resulting in a time complexity of O(max(nums))

    The Canonical solution produces a solution that consumes 5 times more energy than the best solution. This implementation relies on constructing a list with a size proportional to the maximum value in the input array, resulting in a time complexity of O(max(nums)). This increas...

  33. [41]

    Similar to the canonical approach, it uses a list-based method to accumulate points

    Gemini 1.5 Pro produces a solution with energy consumption roughly 7 times greater than the best solution. Similar to the canonical approach, it uses a list-based method to accumulate points. However, the list is statically sized to a fixed upper bound of 10,000, irrespective ...

  34. [42]

    The worst solution is produced by Claude-Sonnet and LLaMA 3.1 70B. While structurally similar to the canonical approach in utilizing a list indexed by element values, their implementations introduce additional redundant multiplications during the computation. These unnecessary...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.