pith. machine review for the scientific record. sign in

arxiv: 1707.02275 · v1 · submitted 2017-07-07 · 💻 cs.CL · cs.AI

Recognition: unknown

A parallel corpus of Python functions and documentation strings for automated code documentation and code generation

Antonio Valerio Miceli Barone , Rico Sennrich
Authors on Pith no claims yet
classification 💻 cs.CL cs.AI
keywords codedocumentationautomatedgenerationparallelareascorpusdata
0
0 comments X
read the original abstract

Automated documentation of programming source code and automated code generation from natural language are challenging tasks of both practical and scientific interest. Progress in these areas has been limited by the low availability of parallel corpora of code and natural language descriptions, which tend to be small and constrained to specific domains. In this work we introduce a large and diverse parallel corpus of a hundred thousands Python functions with their documentation strings ("docstrings") generated by scraping open source repositories on GitHub. We describe baseline results for the code documentation and code generation tasks obtained by neural machine translation. We also experiment with data augmentation techniques to further increase the amount of training data. We release our datasets and processing scripts in order to stimulate research in these areas.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. CodeBLEU: a Method for Automatic Evaluation of Code Synthesis

    cs.SE 2020-09 conditional novelty 7.0

    CodeBLEU improves correlation with human programmer scores on code synthesis tasks by adding syntactic AST matching and semantic data-flow matching to the standard BLEU n-gram approach.

  2. CodeSearchNet Challenge: Evaluating the State of Semantic Code Search

    cs.LG 2019-09 accept novelty 7.0

    Releases a large multi-language code corpus and expert-annotated challenge to benchmark semantic code search.

  3. LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code

    cs.SE 2024-03 unverdicted novelty 6.0

    LiveCodeBench collects 400 recent contest problems to create a contamination-free benchmark evaluating LLMs on code generation and related capabilities like self-repair and execution.

  4. CRUXEval: A Benchmark for Code Reasoning, Understanding and Execution

    cs.SE 2024-01 accept novelty 6.0

    CRUXEval benchmark shows current code models including GPT-4 achieve at most 81% on input and output prediction for short Python functions, exposing gaps not captured by HumanEval.