Can It Edit? Evaluating the Ability of Large Language Models to Follow Code Editing Instructions

1 code implementation • 11 Dec 2023 • Federico Cassano, Luisa Li, Akul Sethi, Noah Shinn, Abby Brennan-Jones, Jacob Ginesin, Edward Berman, George Chakhnashvili, Anton Lozhkov, Carolyn Jane Anderson, Arjun Guha

These are tasks in which the model is provided a block of code and an instruction to modify the code.

Type Prediction With Program Decomposition and Fill-in-the-Type Training

1 code implementation • 25 May 2023 • Federico Cassano, Ming-Ho Yee, Noah Shinn, Arjun Guha, Steven Holtzen

TypeScript and Python are two programming languages that support optional type annotations, which are useful but tedious to introduce and maintain.

Type prediction

Reflexion: Language Agents with Verbal Reinforcement Learning

2 code implementations • NeurIPS 2023 • Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, Shunyu Yao

Large language models (LLMs) have been increasingly used to interact with external environments (e. g., games, compilers, APIs) as goal-driven agents.

Decision Making reinforcement-learning