1 code implementation • 8 Dec 2023 • Anjan Karmakar, Romain Robbes
We find that models that incorporate some structural information (such as GraphCodeBERT) have a better representation of source code characteristics.
1 code implementation • 18 Dec 2022 • Anjan Karmakar, Miltiadis Allamanis, Romain Robbes
To demonstrate the utility of the dataset, we also report results from two empirical studies on our data, ultimately showing that significant work lies ahead in the design of context-aware source code models that can reason over a broader network of source code entities in a software project, the very task that JEMMA is designed to help with.
no code implementations • 6 Dec 2022 • Anjan Karmakar, Julian Aron Prenner, Marco D'Ambros, Romain Robbes
In this work, we evaluate the code synthesis capabilities of the Codex model based on a set of 115 Python problem statements from a popular competitive programming portal: HackerRank.
1 code implementation • IEEE/ACM International Conference on Automated Software Engineering (ASE) 2021 • Anjan Karmakar, Romain Robbes
Pre-trained models of code built on the transformer architecture have performed well on software engineering (SE) tasks such as predictive code generation, code summarization, among others.
no code implementations • 1 Jan 2021 • Anjan Karmakar, Julian Aron Prenner, Miltiadis Allamanis, Romain Robbes
To address this, we present GLUECode, Global and Local Understanding Evaluation of Code, a benchmark of diverse tasks to evaluate machine learning models of source code.