Code Generation
337 papers with code • 17 benchmarks • 43 datasets
Code Generation is an important field to predict explicit code or program structure from multimodal data sources such as incomplete code, programs in another programming language, natural language descriptions or execution examples. Code Generation tools can assist the development of automatic programming tools to improve programming productivity.
Source: Deep Learning for Source Code Modeling and Generation
Image source: Measuring Coding Challenge Competence With APPS
Benchmarks
These leaderboards are used to track progress in Code Generation
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
Libraries
Use these libraries to find Code Generation models and implementations
HumanEval
MBPP
WikiSQL
CodeXGLUE
APPS
DS-1000
Django
Subtasks
Most implemented papers
Programming Puzzles
The dataset is comprehensive in that it spans problems of a range of difficulties and domains, ranging from trivial string manipulation problems, to classic programming puzzles (e. g., Tower of Hanoi), to interview/competitive-programming problems (e. g., dynamic programming), to longstanding open problems in algorithms and mathematics (e. g., factoring).
Asleep at the Keyboard? Assessing the Security of GitHub Copilot's Code Contributions
xqx12/daily-info
•
•
The most notable of these comes in the form of the first self-described `AI pair programmer', GitHub Copilot, a language model trained over open-source GitHub code.
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback
We apply preference modeling and reinforcement learning from human feedback (RLHF) to finetune language models to act as helpful and harmless assistants.
InCoder: A Generative Model for Code Infilling and Synthesis
dpfried/incoder
•
•
Our model is the first generative model that is able to directly perform zero-shot code infilling, which we evaluate on challenging tasks such as type inference, comment generation, and variable re-naming.
AskIt: Unified Programming Interface for Programming with Large Language Models
Developers face decisions regarding the use of LLMs for directly performing tasks within applications as well as for generating and executing code to accomplish these tasks.
Mixtral of Experts
hit-scir/chinese-mixtral-8x7b
•
•
In particular, Mixtral vastly outperforms Llama 2 70B on mathematics, code generation, and multilingual benchmarks.
Latent Predictor Networks for Code Generation
Many language generation tasks require the production of text conditioned on both structured and unstructured inputs.
Bidirectional Attention for SQL Generation
Generating structural query language (SQL) queries from natural language is a long-standing open problem.
Building Language Models for Text with Named Entities
uclanlp/NamedEntityLanguageModel
•
•
ACL 2018
Text in many domains involves a significant amount of named entities.
Structural Language Models of Code
tech-srl/slm-code-generation
•
•
ICML 2020
We introduce a new approach to any-code completion that leverages the strict syntax of programming languages to model a code snippet as a tree - structural language modeling (SLM).
