no code implementations • 18 Feb 2024 • Shraddha Barke, Christian Poelitz, Carina Suzana Negreanu, Benjamin Zorn, José Cambronero, Andrew D. Gordon, Vu Le, Elnaz Nouri, Nadia Polikarpova, Advait Sarkar, Brian Slininger, Neil Toronto, Jack Williams
Large language models (LLMs) are rapidly replacing help forums like StackOverflow, and are especially helpful for non-professional programmers and end users.
no code implementations • 13 Dec 2023 • Mukul Singh, José Cambronero, Sumit Gulwani, Vu Le, Gust Verbruggen
Multi-modality promises to unlock further uses for large language models.
no code implementations • 26 Oct 2023 • Mukul Singh, José Cambronero, Sumit Gulwani, Vu Le, Carina Negreanu, Elnaz Nouri, Mohammad Raza, Gust Verbruggen
Writing such rules is often challenging for users as it requires them to understand and implement the underlying logic.
no code implementations • 26 Oct 2023 • Mukul Singh, José Cambronero, Sumit Gulwani, Vu Le, Carina Negreanu, Gust Verbruggen
Imagine a developer who can only change their last line of code, how often would they have to start writing a function from scratch before it is correct?
no code implementations • 16 Oct 2023 • Ananya Singha, José Cambronero, Sumit Gulwani, Vu Le, Chris Parnin
Inspired by prior work, we generate a collection of self-supervised structural tasks (e. g. navigate to a cell and row; transpose the table) and evaluate the performance differences when using 8 formats.
2 code implementations • 5 Oct 2023 • Tung Phung, Victor-Alexandru Pădurean, Anjali Singh, Christopher Brooks, José Cambronero, Sumit Gulwani, Adish Singla, Gustavo Soares
We investigate the role of generative AI models in providing human tutor-style programming hints to help students resolve errors in their buggy programs.
no code implementations • 2 Oct 2023 • Andrew D. Gordon, Carina Negreanu, José Cambronero, Rasika Chakravarthy, Ian Drosos, Hao Fang, Bhaskar Mitra, Hannah Richardson, Advait Sarkar, Stephanie Simmons, Jack Williams, Ben Zorn
Hence, we are seeing the emergence of tool-assisted experiences to help the user double-check a piece of AI-generated content.
no code implementations • 21 Aug 2023 • Mukul Singh, José Cambronero, Sumit Gulwani, Vu Le, Carina Negreanu, Gust Verbruggen
DataVinci learns regular-expression-based patterns that cover a majority of values in a column and reports values that do not satisfy such patterns as data errors.
no code implementations • 29 Jun 2023 • Tung Phung, Victor-Alexandru Pădurean, José Cambronero, Sumit Gulwani, Tobias Kohn, Rupak Majumdar, Adish Singla, Gustavo Soares
In our work, we systematically evaluate two models, ChatGPT (based on GPT-3. 5) and GPT-4, and compare their performance with human tutors for a variety of scenarios.
no code implementations • 31 Jan 2023 • Harshit Joshi, Abishai Ebenezer, José Cambronero, Sumit Gulwani, Aditya Kanade, Vu Le, Ivan Radiček, Gust Verbruggen
We evaluate FLAME on formula repair, formula completion, and similarity-based formula retrieval.
1 code implementation • 24 Jan 2023 • Tung Phung, José Cambronero, Sumit Gulwani, Tobias Kohn, Rupak Majumdar, Adish Singla, Gustavo Soares
We investigate using LLMs to generate feedback for fixing syntax errors in Python programs, a key scenario in introductory programming.
no code implementations • 29 Sep 2022 • Jialu Zhang, José Cambronero, Sumit Gulwani, Vu Le, Ruzica Piskac, Gustavo Soares, Gust Verbruggen
We propose to use a large language model trained on code, such as Codex, to build an APR system -- MMAPR -- for introductory Python programming assignments.
no code implementations • 24 Aug 2022 • Harshit Joshi, José Cambronero, Sumit Gulwani, Vu Le, Ivan Radicek, Gust Verbruggen
We show that RING can outperform language-specific repair engines for three of these languages.
no code implementations • 11 Aug 2022 • Mukul Singh, José Cambronero, Sumit Gulwani, Vu Le, Carina Negreanu, Mohammad Raza, Gust Verbruggen
Since we are the first to introduce conditional formatting, we compare CORNET to a wide range of symbolic and neural baselines adapted from related domains.