Pith. sign in

REVIEW 2 cited by

A Comparative Study of Code Generation using ChatGPT 3.5 across 10 Programming Languages

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2308.04477 v1 pith:T3LXLRE3 submitted 2023-08-08 cs.SE cs.AI

Alessio Buscemi This is my paper
classification cs.SEcs.AI
keywords codeacrosslanguagesprogrammingchatgptgeneratinggenerationlanguage
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Large Language Models (LLMs) are advanced Artificial Intelligence (AI) systems that have undergone extensive training using large datasets in order to understand and produce language that closely resembles that of humans. These models have reached a level of proficiency where they are capable of successfully completing university exams across several disciplines and generating functional code to handle novel problems. This research investigates the coding proficiency of ChatGPT 3.5, a LLM released by OpenAI in November 2022, which has gained significant recognition for its impressive text generating and code creation capabilities. The skill of the model in creating code snippets is evaluated across 10 various programming languages and 4 different software domains. Based on the findings derived from this research, major unexpected behaviors and limitations of the model have been identified. This study aims to identify potential areas for development and examine the ramifications of automated code generation on the evolution of programming languages and on the tech industry.

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. The Impact of LLM-Assistants on Software Developer Productivity: A Systematic Review and Mapping Study

    cs.SE 2025-07 conditional novelty 6.0 of 10

    A systematic review of 39 studies shows LLM coding assistants generally improve perceived speed and reduce online search, while code quality outcomes remain inconsistent and long-term team effects are understudied.

  2. Toward Inclusive AI-Driven Development: Exploring Gender Differences in Code Generation Tool Interactions

    cs.SE 2025-07 unverdicted novelty 4.0 of 10

    A registered-report style proposal for testing gender differences in how developers interact with AI code generation tools, with no results reported yet.

Pith tools