REVIEW 1 cited by
On Evaluating the Efficiency of Source Code Generated by LLMs
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recent years have seen the remarkable capabilities of large language models (LLMs) for code generation. Different from existing work that evaluate the correctness of the code generated by LLMs, we propose to further evaluate its efficiency. More efficient code can lead to higher performance and execution efficiency of programs and software completed by LLM-assisted programming. First, we evaluate the efficiency of the code generated by LLMs on two benchmarks, HumanEval and MBPP. Then, we choose a set of programming problems from the online judge platform LeetCode to conduct a more difficult evaluation. Finally, we explore several prompts that would enable LLMs to generate more efficient code.
Forward citations
Cited by 1 Pith paper
-
Evaluating the Energy-Efficiency of the Code Generated by LLMs
LLM-generated Python solutions typically consume more energy than canonical human-written solutions, with DeepSeek-v3 and GPT-4o the most efficient LLMs and worst-case gaps near 450 times on certain problems.
Discussion (0). Continue with ORCID to comment.