REVIEW 2 cited by
Recommendations for Datasets for Source Code Summarization
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Source Code Summarization is the task of writing short, natural language descriptions of source code. The main use for these descriptions is in software documentation e.g. the one-sentence Java method descriptions in JavaDocs. Code summarization is rapidly becoming a popular research problem, but progress is restrained due to a lack of suitable datasets. In addition, a lack of community standards for creating datasets leads to confusing and unreproducible research results -- we observe swings in performance of more than 33% due only to changes in dataset design. In this paper, we make recommendations for these standards from experimental results. We release a dataset based on prior work of over 2.1m pairs of Java methods and one sentence method descriptions from over 28k Java projects. We describe the dataset and point out key differences from natural language data, to guide and support future researchers.
Forward citations
Cited by 2 Pith papers
-
Optimizing Datasets for Code Summarization: Is Code-Comment Coherence Enough?
Filtering code summarization training data by code-comment coherence gives no better results than random selection, and halving the training set does not hurt performance.
-
CIDRe: A Reference-Free Multi-Aspect Criterion for Code Comment Quality Measurement
CIDRe is a reference-free, four-part quality score for structured code comments, validated on 840 manually labeled Russian comments and reported to improve comment generation after dataset filtering.
Discussion (0). Continue with ORCID to comment.