Code Generation
322 papers with code • 17 benchmarks • 41 datasets
Code Generation is an important field to predict explicit code or program structure from multimodal data sources such as incomplete code, programs in another programming language, natural language descriptions or execution examples. Code Generation tools can assist the development of automatic programming tools to improve programming productivity.
Source: Deep Learning for Source Code Modeling and Generation
Image source: Measuring Coding Challenge Competence With APPS
Benchmarks
These leaderboards are used to track progress in Code Generation
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Code Generation models and implementations
HumanEval
MBPP
WikiSQL
CodeXGLUE
APPS
DS-1000
Django
Subtasks
Latest papers
DevBench: A Comprehensive Benchmark for Software Development
Recent advancements in large language models (LLMs) have significantly enhanced their coding capabilities.
CleanAgent: Automating Data Standardization with LLM-based Agents
Data standardization is a crucial part in data science life cycle.
Bugs in Large Language Models Generated Code: An Empirical Study
The bug patterns are presented in the form of a taxonomy.
Branch-Train-MiX: Mixing Expert LLMs into a Mixture-of-Experts LLM
Leeroo-AI/mergoo
•
•
We investigate efficient methods for training Large Language Models (LLMs) to possess capabilities in multiple specialized domains, such as coding, math reasoning and world knowledge.
Automatic Generation of Python Programs Using Context-Free Grammars
In recent years, data has emerged as the new gold, serving as a powerful tool for creating intelligent systems.
Text2QR: Harmonizing Aesthetic Customization and Scanning Robustness for Text-Guided QR Code Generation
This approach harnesses the potent generation capabilities of stable-diffusion models, navigating the trade-off between image aesthetics and QR code scannability.
UniSparse: An Intermediate Language for General Sparse Format Customization
The ongoing trend of hardware specialization has led to a growing use of custom data formats when processing sparse workloads, which are typically memory-bound.
Quantifying Contamination in Evaluating Code Generation Capabilities of Language Models
While large language models have achieved remarkable performance on various code generation benchmarks, there have been growing concerns regarding potential contamination of these benchmarks as they may be leaked into pretraining and finetuning data.
IRCoder: Intermediate Representations Make Language Models Robust Multilingual Code Generators
In particular, most mainstream Code-LMs have been pre-trained on source code files alone.
DACO: Towards Application-Driven and Comprehensive Data Analysis via Code Generation
We construct the DACO dataset, containing (1) 440 databases (of tabular data) collected from real-world scenarios, (2) ~2k query-answer pairs that can serve as weak supervision for model training, and (3) a concentrated but high-quality test set with human refined annotations that serves as our main evaluation benchmark.
