ChatGPT vs. DeepSeek: A Comparative Study on AI-Based Code Generation

Md Motaleb Hossen Manik

REVIEW 4 major objections 5 minor 11 references

ChatGPT vs. DeepSeek: A Comparative Study on AI-Based Code Generation

T0 review · 4 major / 5 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper argues that DeepSeek R1 outperforms ChatGPT o1 at producing correct Python code on algorithmic contest problems, often on the first try, while ChatGPT writes shorter, cleaner code.

desk verdict A thin, outcome-selected benchmark whose headline claim about DeepSeek's superiority is not supported by its own numbers. read the letter →

arxiv 2502.18467 v1 pith:IKG6X7MR submitted 2025-01-30 cs.SE cs.AI

Md Motaleb Hossen Manik This is my paper · ORCID

classification cs.SEcs.AI

keywords ChatGPTvsDeepSeekAIinsoftwaredevelopmentcodegenerationAutomatedPythonlargelanguagemodelsCodeforcescorrectness

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish which of two AI assistants writes better Python code, by comparing ChatGPT o1 with DeepSeek R1 on online-judge problems. Its reported result is that DeepSeek produces correct solutions more often, frequently on the first try, so its aggregate correctness score is higher. A reader should care because this is the judgment a developer makes when choosing a coding assistant: correctness with fewer repair cycles saves time, even when the other model writes cleaner and shorter code. The paper also reports that ChatGPT's code had fewer lint issues, comparable or slightly lower execution time and memory, and fewer active lines.

What carries the argument

The central mechanism is an iterative online-judge loop: submit generated code to Codeforces, feed a non-Accepted verdict back to the model as an error message, resubmit up to three times, and score correctness as accepted attempts over total attempts. Correctness is scored as $\frac{\sum C_i}{\sum T_i}$, the number of Accepted verdicts over total submissions, giving 0.5454 for DeepSeek and 0.1875 for ChatGPT. The comparison also uses Pylint/Flake8 issue counts, measured wall-clock time and memory, and active line count, with the correctness loop carrying the paper's main argument.

What would settle it

Submit both models to a fixed, pre-registered set of 50 Codeforces problems of mixed difficulty with the same three-attempt judge loop; if DeepSeek's Accepted rate and average attempts-to-accept are not better than ChatGPT's across the full set, the paper's central claim fails. A smaller check is to rerun the exact seven problems across many fresh sessions, since the current table reports single sessions.

Watch

Extended reading notes

Core claim

The paper's central discovery is that DeepSeek R1, when asked to solve Codeforces-style algorithmic problems in Python, achieved Accepted verdicts more often and with fewer resubmissions than ChatGPT o1. On the seven retained contest problems, DeepSeek solved cases that ChatGPT could not fix in three attempts, and the paper's aggregate correctness scores put DeepSeek at 0.5454 versus 0.1875 for ChatGPT. The paper also reports that on ten simple coding tasks ChatGPT posted fewer lint issues, shorter active code, and comparable or slightly lower time and memory, so the advantage is not a clean sweep. In the author's framing, DeepSeek has a potential advantage in tasks demanding precise algorithmic implementation, while ChatGPT remains competitive on style and conciseness.

Load-bearing premise

The load-bearing premise is that the seven Codeforces problems kept for the study represent Python coding well enough to back a general claim that DeepSeek is better; the benchmark was narrowed after manual tests produced no difference, so those seven problems could be precisely the ones where DeepSeek happens to shine.

Editorial extensions

If this is right

DeepSeek's higher correctness score means an algorithmic coding workflow with judge feedback would need fewer model round-trips.
ChatGPT's lower lint-issue counts and shorter code remain advantages once correctness is not the bottleneck.
Both models can use judge error messages to repair their code, but DeepSeek converted those messages into Accepted solutions more often in this study.
The correctness and style comparisons rest on different problem sets: seven Codeforces problems for correctness, ten simpler problems for style, time, memory, and lines.
The reported efficiency scores, 0.62 for DeepSeek versus 0.51 for ChatGPT, indicate that the advantage persists when time to an accepted solution is factored in.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial extension: because the seven-problem correctness set was chosen only after manual tests failed to differentiate the models, the numeric gap should be treated as a hypothesis-generating signal until a larger pre-registered sample reproduces it.
Editorial extension: the practical gap may be larger than raw judge outcomes suggest, because a first-try Accepted solution saves developer time and re-prompting effort that a third-try fix does not.
Editorial extension: the same protocol on non-contest or non-Python tasks, or on newer model versions, could plausibly reverse the ranking, so the result is scoped to algorithmic Python contest problems.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Hi — quick take. This is a small head-to-head that does not support its abstract. The only new thing is the specific pair (ChatGPT o1 vs DeepSeek R1) on a handful of Codeforces problems. That is a legitimate measurement, but it is too thin and too outcome-dependent to justify the general claim that DeepSeek is superior at Python code generation.

What it does well: it's readable, the repair-loop protocol (submit, feed errors back, up to three attempts) is sensible, and the author cites relevant prior work (EvalPlus, DevGPT, the Copilot comparisons). The limitation note in Section V is honest about missing data and server issues. The raw result — 6/7 accepted for DeepSeek vs 3/7 for ChatGPT on this set — is what it is.

The soft spots are load-bearing. Section III says manual test cases failed to show differences, so the study moved to Codeforces and then kept seven problems. That means the benchmark was chosen after the fact because it produced differences. No pre-registration, no independence, no rationale for the seven. And even on those numbers, the discordant pairs are 3-0, so the two-sided McNemar p-value is 0.25 — not significant. The correctness metric is a raw ratio with no uncertainty. Table I is garbled, Table II mixes ten trivial tasks into a table that is supposed to show code quality, and there are no prompts or artifacts, so no one can reproduce or extend it. The conclusion in Section VI far outruns the evidence.

Who is this for? Maybe someone compiling anecdotal comparisons of coding assistants, or a class discussion on benchmark design. It is not a reliable basis for choosing a tool. The author seems honest about limitations, and the thinking is not incoherent, but the central claim just isn't supported.

My recommendation: I would not send this to a serious peer review as is. It deserves to be reformulated as a small replication note with narrow claims, or rejected. If you're tracking this space, treat it as a cautionary example of outcome-dependent sampling, not as evidence.

Referee Report

4 major / 5 minor

Summary. The paper compares ChatGPT (o1) and DeepSeek (R1) on Python code generation. Correctness is evaluated by submitting generated solutions to Codeforces online judge problems, with up to three attempts per problem on a set of seven contest tasks. Code quality, efficiency, and conciseness are evaluated on ten simple programming tasks using Pylint/Flake8, execution time and memory, and active lines of code. The paper reports that DeepSeek achieves higher correctness (0.5454 vs. 0.1875 on a custom metric), often on the first or second attempt, while ChatGPT produces fewer lint issues, shorter code, and comparable or slightly better memory/time usage. The conclusion states that DeepSeek exhibits superior correctness in Python code generation and suggests an advantage in algorithmic problem-solving.

Significance. If the central claim were well supported, a head-to-head comparison of OpenAI o1 and DeepSeek R1 on algorithmic code generation would be useful for practitioners and researchers. The use of an external judge (Codeforces) for correctness is a sound idea, and the paper reports detailed per-problem outcomes in Table I, which is commendable. However, the study as presented does not provide a valid basis for the claimed general superiority. The benchmark was selected after preliminary tests failed to show differences, the sample is tiny, no statistical testing is performed, and the definitions of the evaluation metrics are problematic. The paper is therefore not yet a reliable empirical comparison, despite the transparency of its raw results.

major comments (4)

[III (Test cases)] The selection of the Codeforces test set is outcome-dependent. The manuscript states that manually created test cases 'failed to reveal significant differences' and that the study consequently switched to Codeforces, which 'effectively highlighted the differences.' This means the seven retained problems were not chosen independently of the observed results, so the correctness comparison cannot support a general claim about model superiority. No evidence is given that the seven problems are representative of Python coding tasks or algorithmic problem-solving more broadly.
[Table I and Section IV-B, Eq. (1)] The correctness evidence is statistically weak. With seven paired problems, counting an Accepted verdict at any attempt gives DeepSeek 6/7 and ChatGPT 3/7, with three discordant pairs all favoring DeepSeek; the exact two-sided McNemar p-value is 0.25. The paper instead reports a custom metric, the ratio of accepted attempts to total attempts (0.5454 vs. 0.1875), which is not a per-problem success rate and is directly affected by the arbitrary three-attempt cap. No confidence intervals or significance tests are provided, so even the direction of the effect is not reliably established.
[IV-B, Eq. (2)] The efficiency metric is not meaningful as defined. Efficiency = sum of accepted attempts divided by sum of execution times for accepted attempts gives units of attempts per millisecond and mixes counts with durations. It is also computed only over accepted attempts, which creates a selection bias that is not discussed. The subsequent claim in the abstract and Section VI that 'both models showed almost similar efficiency' is based on Table II of ten simple tasks, not on the Codeforces problems used for the correctness comparison, yet the conclusion combines these different evaluations without acknowledging the mismatch.
[V (Discussion and Limitations)] The paper admits lacking 'detailed information regarding resource constraints, server issues for DeepSeek, and data sources,' but these omissions are load-bearing for an empirical comparison. Without the exact prompts, model API settings (temperature, top-p, seed), date of runs, and the full set of generated code and judge verdicts, the experiment cannot be reproduced or independently verified. The absence of these artifacts is particularly serious given the small sample size and the outcome-dependent test-case selection.

minor comments (5)

[Table I] The table header contains spelling errors ('REGARIDG', 'DeekSeek') and the column alignment is confusing because the 'Attempt 1/2/3' labels are not visually tied to the per-model columns; please reformat for clarity.
[Table II] The entry '0..9' for DeepSeek's reverse linked list time is a typo and should be '0.9' or similar; also, the issue counts are not defined (what counts as one 'issue' in Pylint/Flake8?).
[Fig. 1] The figure label says 'comparion' and the bars are not annotated with numerical values or error bars; please correct the label and add the underlying numbers.
[IV-C and Table III] Table III is a subjective feature comparison (e.g., 'Advanced contextual understanding') that is not derived from the experimental results; please mark it clearly as an interpretive summary or remove it.
[References] Reference [5] contains placeholder DOI text ('10.1145/nnnnnnn.nnnnnnn') and the reference list should be checked for formatting consistency.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the correctness comparison is a direct empirical measurement against the external Codeforces judge, with no fitted parameters or self-citation chain carrying the conclusion.

full rationale

The paper's central claim is an empirical benchmark result: DeepSeek and ChatGPT generated Python solutions to seven Codeforces problems, and the submissions were graded by the external online-judge verdicts (Accepted/Wrong/Runtime). The correctness metric (Correctness = sum of accepted attempts / sum of total attempts, Section IV.B.1) is an operationalization of the raw verdict counts (3/16 = 0.1875 for ChatGPT, 6/11 = 0.5454 for DeepSeek), and the abstract's conclusion straightforwardly restates that computed comparison. There is no equation that defines the conclusion into existence: the verdicts come from an independent judge, not from the paper's own definitions, so the derivation chain is not self-definitional. No parameter is fitted and then renamed as a prediction; the models' outputs are generated and judged directly. The paper cites DeepSeek-AI technical reports (references [10], [11]) only as background on the model family, not as load-bearing evidence for the comparison, and the author does not rely on a self-citation chain or an imported uniqueness theorem. The weaknesses that do exist are validity concerns rather than circularity: Section III (Test cases) states the study switched to Codeforces only after manually created test cases 'failed to reveal significant differences,' so the benchmark was selected partly because it produced divergence, and Section V admits missing data sources and resource constraints. Those are selection-bias and reproducibility threats to the generalization of 'DeepSeek consistently exhibited superior performance,' but they do not make any step of the derivation equivalent to its own input. Because the correctness verdicts are externally grounded, the appropriate circularity score is 0.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The comparison depends on the hand-picked benchmark, the three-attempt limit, and unvalidated metric definitions; none are justified independently or released as data.

free parameters (3)

Maximum attempts per problem = 3
The protocol stops after three submissions (Section III.3.a); the correctness metric divides accepted attempts by total attempts, so this cutoff directly shapes the score without justification.
Number of contest problems used = 7
Only seven Codeforces problems were retained for correctness (Table I), and the initial manual test set was discarded after it 'failed to reveal significant differences' (Section III.ii).
Number of simple tasks used = 10
Ten simple self-generated problems were used for quality, efficiency, and conciseness (Section IV.A.2 and Table II) with no source or inclusion criteria.

assumptions (4)

domain assumption Online judge 'Accepted' verdicts are a valid proxy for code correctness.
The correctness comparison treats Codeforces verdicts as ground truth (Section III.3.a) and then generalizes to Python code generation broadly.
domain assumption Pylint and Flake8 issue counts measure code quality and readability.
The paper defines readability as the inverse of issues per generated code (Section IV.B.3), equating static-analysis warnings with readability without validation.
domain assumption The two models were prompted and run under comparable conditions.
No prompt texts, API parameters, dates, or environment details are reported, so the head-to-head comparison assumes equivalent input and execution (Section III).
ad hoc to paper The three-attempt cap does not bias the comparison.
The cap at three attempts and the metric sum(C_i)/sum(T_i) favor whichever model converges earlier; this is an unstated modeling choice (Section IV.B.1).

how reviews work

0 comments

Cite this review

Pith. "Pith review of ChatGPT vs. DeepSeek: A Comparative Study on AI-Based Code Generation." pith.science (2026). https://pith.science/paper/IKG6X7MR

@misc{pith2026250218467,
  author       = {Pith},
  title        = {Pith review of: ChatGPT vs. DeepSeek: A Comparative Study on AI-Based Code Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IKG6X7MR}},
  note         = {Machine review of arXiv:2502.18467}
}

read the original abstract

Background: AI-powered code generation, fueled by Large Language Models (LLMs), is revolutionizing software development. Models like OpenAI's Codex and GPT-4, alongside DeepSeek, leverage vast code and natural language datasets. However, ensuring code quality, correctness, and managing complex tasks remains challenging, necessitating thorough evaluation. Methodology: This research compares ChatGPT (version o1) and DeepSeek (version R1) for Python code generation using online judge coding challenges. It evaluates correctness (online judge verdicts, up to three attempts), code quality (Pylint/Flake8), and efficiency (execution time/memory usage). Results: DeepSeek demonstrated higher correctness, particularly on algorithmic tasks, often achieving 'Accepted' on the first attempt. ChatGPT sometimes requires multiple attempts or failures. ChatGPT encountered fewer issues, used comparable or slightly less memory, consumed less execution times and wrote fewer lines of code. Conclusion: DeepSeek exhibited superior correctness in Python code generation, often requiring fewer attempts, suggesting an advantage in algorithmic problem-solving. Both models showed almost similar efficiency in execution time and memory use. Finally, this research provides insights for developers choosing AI coding assistants and informs future AI-driven software development research.

Figures

Figures reproduced from arXiv: 2502.18467 by the authors.

**Figure 1.** Metrics value comparion for ChatGPT and DeepSeekA TABLE III. A COMPARISON BETWEEN CHATGPT AND DEEPSEEK Feature ChatGPT DeepSeek Code Generation Strong generalpurpose code generation Specialized code generation focus Programming Languages Wide range of languages Growing language support Code Completion Supports code completion Optimized for code completion Contextual Awareness Good contextual understanding Advanced … view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

11 extracted references · 4 canonical work pages

[1]

Investigating Code Generation Performance of ChatGPT with Crowdsourcing Social Data,

Y. Feng, S. Vanam, M. Cherukupally, W. Zheng, M. Qiu, and H. Chen, “Investigating Code Generation Performance of ChatGPT with Crowdsourcing Social Data,” Proceedings - International Computer Software and Applications Conference, vol. 2023-June, pp. 876–885, 2023, doi: 10.1109/COMPSAC57700.2023.00117

arXiv 2023
[2]

Extending the Frontier of ChatGPT: Code Generation and Debugging,

F. A. Sakib, S. H. Khan, and A. H. M. R. Karim, “Extending the Frontier of ChatGPT: Code Generation and Debugging,” Jul. 2023, doi: 10.1109/ICECET61485.2024.10698405

arXiv 2023
[3]

ChatGPT Code Detection: Techniques for Uncovering the Source of Code,

M. Oedingen, R. C. Engelhardt, R. Denz, M. Hammer, and W. Konen, “ChatGPT Code Detection: Techniques for Uncovering the Source of Code,” AI, vol. 5, no. 3, pp. 1066 –1094, May 2024, doi: 10.3390/ai5030053

work page doi:10.3390/ai5030053 2024
[4]

Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation,

J. Liu, C. S. Xia, Y. Wang, and L. ZHANG, “Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation,” Adv Neural Inf Process Syst , vol. 36, pp. 21558 –21572, Dec. 2023, Accessed: Jan. 29, 2025. [Online]. Available: https://github.com/evalplus/evalplus

work page 2023
[5]

Programming with AI: Evaluating ChatGPT, Gemini, AlphaCode, and GitHub Copilot for Programmers,

M. K. Siam, H. Gu, and J. Q. Cheng, “Programming with AI: Evaluating ChatGPT, Gemini, AlphaCode, and GitHub Copilot for Programmers,” ICCA â•Ž24, September 17â•ﬁ18, 2024 , vol. 1, Nov. 2024, doi: 10.1145/nnnnnnn.nnnnnnn

doi:10.1145/nnnnnnn.nnnnnnn 2024
[6]

AI Writes, We Analyze: The ChatGPT Python Code Saga,

M. F. Rabbi, A. Champa, M. Zibran, and M. R. Islam, “AI Writes, We Analyze: The ChatGPT Python Code Saga,” Proceedings - 2024 IEEE/ACM 21st International Conference on Mining Software Repositories, MSR 2024 , pp. 177 –181, 2024, doi: 10.1145/3643991.3645076

arXiv 2024
[7]

Refining ChatGPT-Generated Code: Characterizing and Mitigating Code Quality Issues,

Y. Liu et al., “Refining ChatGPT-Generated Code: Characterizing and Mitigating Code Quality Issues,” ACM Transactions on Software Engineering and Methodology , vol. 33, no. 5, p. 26, Jun. 2024, doi: 10.1145/3643674/ASSET/CDFC8237-C19F-456D-AF76- 89EAA7DDE608/ASSETS/GRAPHIC/TOSEM-2023-0226- F12.JPG

work page doi:10.1145/3643674/asset/cdfc8237-c19f-456d-af76- 2024
[8]

Evaluating the Code Quality of AI -Assisted Code Generation Tools: An Empirical Study on GitHub Copilot, Amazon CodeWhisperer, and ChatGPT,

B. Yetiştiren, I. Özsoy, M. Ayerdem, and E. Tüzün, “Evaluating the Code Quality of AI -Assisted Code Generation Tools: An Empirical Study on GitHub Copilot, Amazon CodeWhisperer, and ChatGPT,” Apr. 2023, Accessed: Jan. 29, 2025. [Online]. Available: https://arxiv.org/abs/2304.10778v2

arXiv 2023

Show all 11 references

[9]

Coding with AI: How Are Tools Like ChatGPT Being Used by Students in Foundational Programming Courses,

A. Ghimire and J. Edwards, “Coding with AI: How Are Tools Like ChatGPT Being Used by Students in Foundational Programming Courses,” Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 14830 ...

2024 doi
[10]

DeepSeek -Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence,

DeepSeek-AI et al. , “DeepSeek -Coder-V2: Breaking the Barrier of Closed-Source Models in Code Intelligence,” Jun. 2024, Accessed: Jan. 29, 2025. [Online]. Available: https://arxiv.org/abs/2406.11931v1

2024 arXiv
[11]

DeepSeek-VL: Towards Real -World Vision-Language Understanding,

H. Lu et al., “DeepSeek-VL: Towards Real -World Vision-Language Understanding,” Mar. 2024, Accessed: Jan. 29, 2025. [Online]. Available: https://arxiv.org/abs/2403.05525v2

2024 arXiv

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.

[1] [1]

Investigating Code Generation Performance of ChatGPT with Crowdsourcing Social Data,

Y. Feng, S. Vanam, M. Cherukupally, W. Zheng, M. Qiu, and H. Chen, “Investigating Code Generation Performance of ChatGPT with Crowdsourcing Social Data,” Proceedings - International Computer Software and Applications Conference, vol. 2023-June, pp. 876–885, 2023, doi: 10.1109/COMPSAC57700.2023.00117

arXiv 2023

[2] [2]

Extending the Frontier of ChatGPT: Code Generation and Debugging,

F. A. Sakib, S. H. Khan, and A. H. M. R. Karim, “Extending the Frontier of ChatGPT: Code Generation and Debugging,” Jul. 2023, doi: 10.1109/ICECET61485.2024.10698405

arXiv 2023

[3] [3]

ChatGPT Code Detection: Techniques for Uncovering the Source of Code,

M. Oedingen, R. C. Engelhardt, R. Denz, M. Hammer, and W. Konen, “ChatGPT Code Detection: Techniques for Uncovering the Source of Code,” AI, vol. 5, no. 3, pp. 1066 –1094, May 2024, doi: 10.3390/ai5030053

work page doi:10.3390/ai5030053 2024

[4] [4]

Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation,

J. Liu, C. S. Xia, Y. Wang, and L. ZHANG, “Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation,” Adv Neural Inf Process Syst , vol. 36, pp. 21558 –21572, Dec. 2023, Accessed: Jan. 29, 2025. [Online]. Available: https://github.com/evalplus/evalplus

work page 2023

[5] [5]

Programming with AI: Evaluating ChatGPT, Gemini, AlphaCode, and GitHub Copilot for Programmers,

M. K. Siam, H. Gu, and J. Q. Cheng, “Programming with AI: Evaluating ChatGPT, Gemini, AlphaCode, and GitHub Copilot for Programmers,” ICCA â•Ž24, September 17â•ﬁ18, 2024 , vol. 1, Nov. 2024, doi: 10.1145/nnnnnnn.nnnnnnn

doi:10.1145/nnnnnnn.nnnnnnn 2024

[6] [6]

AI Writes, We Analyze: The ChatGPT Python Code Saga,

M. F. Rabbi, A. Champa, M. Zibran, and M. R. Islam, “AI Writes, We Analyze: The ChatGPT Python Code Saga,” Proceedings - 2024 IEEE/ACM 21st International Conference on Mining Software Repositories, MSR 2024 , pp. 177 –181, 2024, doi: 10.1145/3643991.3645076

arXiv 2024

[7] [7]

Refining ChatGPT-Generated Code: Characterizing and Mitigating Code Quality Issues,

Y. Liu et al., “Refining ChatGPT-Generated Code: Characterizing and Mitigating Code Quality Issues,” ACM Transactions on Software Engineering and Methodology , vol. 33, no. 5, p. 26, Jun. 2024, doi: 10.1145/3643674/ASSET/CDFC8237-C19F-456D-AF76- 89EAA7DDE608/ASSETS/GRAPHIC/TOSEM-2023-0226- F12.JPG

work page doi:10.1145/3643674/asset/cdfc8237-c19f-456d-af76- 2024

[8] [8]

Evaluating the Code Quality of AI -Assisted Code Generation Tools: An Empirical Study on GitHub Copilot, Amazon CodeWhisperer, and ChatGPT,

B. Yetiştiren, I. Özsoy, M. Ayerdem, and E. Tüzün, “Evaluating the Code Quality of AI -Assisted Code Generation Tools: An Empirical Study on GitHub Copilot, Amazon CodeWhisperer, and ChatGPT,” Apr. 2023, Accessed: Jan. 29, 2025. [Online]. Available: https://arxiv.org/abs/2304.10778v2

arXiv 2023