Neural metrics for evaluating code comments are unreliable for multilingual output, often scoring random noise as high as real generated comments.
Automatic Code Documentation Generation Using GPT-3
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Source code documentation is an important artifact for efficient software development. Code documentation could greatly benefit from automation since manual documentation is often labouring, resource and time-intensive. In this paper, we employed Codex for automatic code documentation creation. Codex is a GPT-3 based model pre-trained on both natural and programming languages. We find that Codex outperforms existing techniques even with basic settings like one-shot learning (i.e., providing only one example for training). Codex achieves an overall BLEU score of 20.6 for six different programming languages (11.2% improvement over earlier state-of-the-art techniques). Thus, Codex shows promise and warrants in-depth future studies for automatic code documentation generation to support diverse development tasks.
citation-role summary
baseline 1
citation-polarity summary
fields
cs.SE 1years
2025 1verdicts
CONDITIONAL 1roles
baseline 1polarities
baseline 1representative citing papers
citing papers explorer
-
A Qualitative Investigation into LLM-Generated Multilingual Code Comments and Automatic Evaluation Metrics
Neural metrics for evaluating code comments are unreliable for multilingual output, often scoring random noise as high as real generated comments.