Search Results for author: Wen-Ding Li

Found 5 papers, 0 papers with code

LiveCodeBench: Holistic and Contamination Free Evaluation of Large Language Models for Code

no code implementations12 Mar 2024 Naman jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, Ion Stoica

Large Language Models (LLMs) applied to code-related applications have emerged as a prominent field, attracting significant interest from both academia and industry.

Code Generation

The Counterfeit Conundrum: Can Code Language Models Grasp the Nuances of Their Incorrect Generations?

no code implementations29 Feb 2024 Alex Gu, Wen-Ding Li, Naman jain, Theo X. Olausson, Celine Lee, Koushik Sen, Armando Solar-Lezama

In this work, we focus on these counterfeit samples: programs sampled from a language model that 1) have a high enough log-probability to be generated at a moderate temperature and 2) pass weak correctness checks.

Code Generation Language Modelling

Natural Language to Code Generation in Interactive Data Science Notebooks

no code implementations19 Dec 2022 Pengcheng Yin, Wen-Ding Li, Kefan Xiao, Abhishek Rao, Yeming Wen, Kensen Shi, Joshua Howland, Paige Bailey, Michele Catasta, Henryk Michalewski, Alex Polozov, Charles Sutton

To measure the performance of AI pair programmers that automatically synthesize programs for those tasks given natural language (NL) intents from users, we build ARCADE, a benchmark of 1082 code generation problems using the pandas data analysis framework in data science notebooks.

Code Generation Language Modelling

Toward Trustworthy Neural Program Synthesis

no code implementations29 Sep 2022 Darren Key, Wen-Ding Li, Kevin Ellis

We develop an approach to estimate the probability that a program sampled from a large language model is correct.

Language Modelling Large Language Model +1

Cannot find the paper you are looking for? You can Submit a new open access paper.