Shellcode_IA32: A Dataset for Automatic Shellcode Generation

We take the first step to address the task of automatically generating shellcodes, i.e., small pieces of code used as a payload in the exploitation of a software vulnerability, starting from natural language comments. We assemble and release a novel dataset (Shellcode_IA32), consisting of challenging but common assembly instructions with their natural language descriptions. We experiment with standard methods in neural machine translation (NMT) to establish baseline performance levels on this task.

PDF Abstract ACL (NLP4Prog) 2021 PDF ACL (NLP4Prog) 2021 Abstract

Datasets


Introduced in the Paper:

Shellcode_IA32

Results from the Paper


Task Dataset Model Metric Name Metric Value Global Rank Benchmark
Code Generation Shellcode_IA32 LSTM-based Sequence to Sequence BLEU-4 62.97 # 3
Exact Match Accuracy 51.55 # 3

Methods