CoDesc: A Large Code-Description Parallel Dataset

Translation between natural language and source code can help software development by enabling developers to comprehend, ideate, search, and write computer programs in natural language. Despite growing interest from the industry and the research community, this task is often difficult due to the lack of large standard datasets suitable for training deep neural models, standard noise removal methods, and evaluation benchmarks. This leaves researchers to collect new small-scale datasets, resulting in inconsistencies across published works. In this study, we present CoDesc -- a large parallel dataset composed of 4.2 million Java methods and natural language descriptions. With extensive analysis, we identify and remove prevailing noise patterns from the dataset. We demonstrate the proficiency of CoDesc in two complementary tasks for code-description pairs: code summarization and code search. We show that the dataset helps improve code search by up to 22\% and achieves the new state-of-the-art in code summarization. Furthermore, we show CoDesc's effectiveness in pre-training--fine-tuning setup, opening possibilities in building pretrained language models for Java. To facilitate future research, we release the dataset, a data processing tool, and a benchmark at \url{}.

PDF Abstract


Introduced in the Paper:


Used in the Paper:

CodeSearchNet CONCODE Funcom

Results from the Paper

Task Dataset Model Metric Name Metric Value Global Rank Result Benchmark
Code Search CoDesc Self-attention Test MRR 0.839 # 1
Source Code Summarization CoDesc Transformer BLEU-4 45.89 # 1
Code Search CoDesc RNN Test MRR 0.766 # 3
Code Search CoDesc NBOW Test MRR 0.812 # 2


No methods listed for this paper. Add relevant methods here