1 code implementation • 8 Dec 2023 • Julian Aron Prenner, Romain Robbes
Our results indicate that overall repair success increases with the size of the local context (albeit not for all bug types) and confirm the common practice that roughly 50-60% of the input window should be used for context leading the bug.
no code implementations • 3 Apr 2023 • Julian Aron Prenner, Romain Robbes
With this dataset we follow several goals: we want to lift Neural Program Repair beyond fully static code representations, foster the use of execution-based features and, by including several different languages, counterbalance the predominance of Java in the current landscape of APR datasets and benchmarks.
no code implementations • 6 Dec 2022 • Anjan Karmakar, Julian Aron Prenner, Marco D'Ambros, Romain Robbes
In this work, we evaluate the code synthesis capabilities of the Codex model based on a set of 115 Python problem statements from a popular competitive programming portal: HackerRank.
no code implementations • 1 Jan 2021 • Anjan Karmakar, Julian Aron Prenner, Miltiadis Allamanis, Romain Robbes
To address this, we present GLUECode, Global and Local Understanding Evaluation of Code, a benchmark of diverse tasks to evaluate machine learning models of source code.