Code Generation
327 papers with code • 17 benchmarks • 43 datasets
Code Generation is an important field to predict explicit code or program structure from multimodal data sources such as incomplete code, programs in another programming language, natural language descriptions or execution examples. Code Generation tools can assist the development of automatic programming tools to improve programming productivity.
Source: Deep Learning for Source Code Modeling and Generation
Image source: Measuring Coding Challenge Competence With APPS
Benchmarks
These leaderboards are used to track progress in Code Generation
| Trend | Dataset | Best Model | Paper | Code | Compare |
|---|---|---|---|---|---|
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
|||||
|
Libraries
Use these libraries to find Code Generation models and implementations
HumanEval
MBPP
WikiSQL
CodeXGLUE
APPS
DS-1000
Django
Subtasks
Latest papers with no code
VoicePilot: Harnessing LLMs as Speech Interfaces for Physically Assistive Robots
Physically assistive robots present an opportunity to significantly increase the well-being and independence of individuals with motor impairments or other forms of disability who are unable to complete activities of daily living.
CodeEditorBench: Evaluating Code Editing Capability of Large Language Models
Large Language Models (LLMs) for code are rapidly evolving, with code editing emerging as a critical capability.
An Investigation into Misuse of Java Security APIs by Large Language Models
We employ both automated and manual approaches to effectively detect security API misuse in the code generated by ChatGPT for these tasks.
Testing the Effect of Code Documentation on Large Language Model Code Understanding
Large Language Models (LLMs) have demonstrated impressive abilities in recent years with regards to code generation and understanding.
Exploring and Evaluating Hallucinations in LLM-Powered Code Generation
The rise of Large Language Models (LLMs) has significantly advanced many applications on software engineering tasks, particularly in code generation.
The Larger the Better? Improved LLM Code-Generation via Budget Reallocation
On the other hand, in scenarios where unit-tests are unavailable, a ranking-based selection of candidates from the smaller model falls short of the performance of a single output from larger ones.
CodeBenchGen: Creating Scalable Execution-based Code Generation Benchmarks
We will release the code of both the framework and the dataset upon acceptance.
A Survey of using Large Language Models for Generating Infrastructure as Code
Automation of IaC is a necessity in the present conditions of the Industry and in this survey, we study the feasibility of applying Large Language Models (LLM) to address this problem.
DataAgent: Evaluating Large Language Models' Ability to Answer Zero-Shot, Natural Language Queries
Conventional processes for analyzing datasets and extracting meaningful information are often time-consuming and laborious.
MAGIS: LLM-Based Multi-Agent Framework for GitHub Issue Resolution
Motivated by the empirical findings, we propose a novel LLM-based Multi-Agent framework for GitHub Issue reSolution, MAGIS, consisting of four kinds of agents customized for the software evolution: Manager, Repository Custodian, Developer, and Quality Assurance Engineer agents.
