Search Results for author:
Found 7 papers, 4 papers with code
Top Leaderboard Ranking = Top Coding Proficiency, Always? EvoEval: Evolving Coding Benchmarks via LLM
Such limitations inevitably lead us to inquire: Is the leaderboard performance on existing benchmarks reliable and comprehensive enough to measure the program synthesis ability of LLMs?
Copiloting the Copilots: Fusing Large Language Models with Completion Engines for Automated Program Repair
Therefore, we propose Repilot, a general code generation framework to further copilot the AI "copilots" (i. e., LLMs) by synthesizing more valid patches during the repair process.
Fuzz4All: Universal Fuzzing with Large Language Models
Moreover, the inputs generated by existing fuzzers are often limited to specific features of the input language, and thus can hardly reveal bugs related to other or new features.
Is Your Code Generated by ChatGPT Really Correct? Rigorous Evaluation of Large Language Models for Code Generation
While EvalPlus is general, we extend the test-cases of the popular HumanEval benchmark by 80x to build HumanEval+.
Keep the Conversation Going: Fixing 162 out of 337 bugs for $0.42 each using ChatGPT
For earlier patches that failed to pass all tests, we combine the incorrect patches with their corresponding relevant test failure information to construct a new prompt for the LLM to generate the next patch.
Revisiting the Plastic Surgery Hypothesis via Large Language Models
Traditional APR tools have largely leveraged the plastic surgery hypothesis by designing manual or heuristic-based approaches to exploit such existing code ingredients.
Conversational Automated Program Repair
As such, we leverage the long-term context window of LLMs to not only avoid generating previously incorrect patches but also incorporate validation feedback to help the model understand the semantic meaning of the program under test.
