Neural Code Search Revisited: Enhancing Code Snippet Retrieval through Natural Language Intent

27 Aug 2020 Geert Heyman Tom Van Cutsem

In this work, we propose and study annotated code search: the retrieval of code snippets paired with brief descriptions of their intent using natural language queries. On three benchmark datasets, we investigate how code retrieval systems can be improved by leveraging descriptions to better capture the intents of code snippets... (read more)

PDF Abstract

Datasets


TASK DATASET MODEL METRIC NAME METRIC VALUE GLOBAL RANK RESULT BENCHMARK
Annotated Code Search PACS-CoNaLa Ensemble:USE-tuned+NCS MRR 0.351 # 1
Annotated Code Search PACS-CoNaLa USE-tuned MRR 0.340 # 2
Annotated Code Search PACS-CoNaLa USE MRR 0.181 # 3
Annotated Code Search PACS-CoNaLa NCS MRR 0.167 # 4
Annotated Code Search PACS-SO-DS USE-tuned MRR 0.304 # 2
Annotated Code Search PACS-SO-DS USE MRR 0.244 # 3
Annotated Code Search PACS-SO-DS Ensemble:USE-tuned+NCS MRR 0.323 # 1
Annotated Code Search PACS-SO-DS NCS MRR 0.113 # 4
Annotated Code Search PACS-StaQC-py NCS MRR 0.030 # 4
Annotated Code Search PACS-StaQC-py USE-tuned MRR 0.117 # 2
Annotated Code Search PACS-StaQC-py USE MRR 0.104 # 3
Annotated Code Search PACS-StaQC-py Ensemble:USE-tuned+NCS MRR 0.126 # 1

Methods used in the Paper


METHOD TYPE
🤖 No Methods Found Help the community by adding them if they're not listed; e.g. Deep Residual Learning for Image Recognition uses ResNet