Learning to Model the World with Language

no code implementations • 31 Jul 2023 • Jessy Lin, Yuqing Du, Olivia Watkins, Danijar Hafner, Pieter Abbeel, Dan Klein, Anca Dragan

To interact with humans in the world, agents need to understand the diverse types of language that people use, relate them to the visual world, and act based on them.

Future prediction General Knowledge +1

Decision-Oriented Dialogue for Human-AI Collaboration

1 code implementation • 31 May 2023 • Jessy Lin, Nicholas Tomlin, Jacob Andreas, Jason Eisner

In each of these settings, AI assistants and users have disparate abilities that they must combine to arrive at the best decision: assistants can access and process large amounts of information, while users have preferences and constraints external to the system.

UniMASK: Unified Inference in Sequential Decision Problems

1 code implementation • 20 Nov 2022 • Micah Carroll, Orr Paradise, Jessy Lin, Raluca Georgescu, Mingfei Sun, David Bignell, Stephanie Milani, Katja Hofmann, Matthew Hausknecht, Anca Dragan, Sam Devlin

Randomly masking and predicting word tokens has been a successful approach in pre-training language models for a variety of downstream tasks.

Decision Making

Automatic Correction of Human Translations

1 code implementation • NAACL 2022 • Jessy Lin, Geza Kovacs, Aditya Shastry, Joern Wuebker, John DeNero

We show that human errors in TEC exhibit a more diverse range of errors and far fewer translation fluency errors than the MT errors in automatic post-editing datasets, suggesting the need for dedicated TEC models that are specialized to correct human errors.

Automatic Post-Editing Translation

Towards Flexible Inference in Sequential Decision Problems via Bidirectional Transformers

no code implementations • 28 Apr 2022 • Micah Carroll, Jessy Lin, Orr Paradise, Raluca Georgescu, Mingfei Sun, David Bignell, Stephanie Milani, Katja Hofmann, Matthew Hausknecht, Anca Dragan, Sam Devlin

Randomly masking and predicting word tokens has been a successful approach in pre-training language models for a variety of downstream tasks.

Decision Making Offline RL

InCoder: A Generative Model for Code Infilling and Synthesis

3 code implementations • 12 Apr 2022 • Daniel Fried, Armen Aghajanyan, Jessy Lin, Sida Wang, Eric Wallace, Freda Shi, Ruiqi Zhong, Wen-tau Yih, Luke Zettlemoyer, Mike Lewis

Our model is the first generative model that is able to directly perform zero-shot code infilling, which we evaluate on challenging tasks such as type inference, comment generation, and variable re-naming.

Code Generation Comment Generation +1

Inferring Rewards from Language in Context

1 code implementation • ACL 2022 • Jessy Lin, Daniel Fried, Dan Klein, Anca Dragan

In classic instruction following, language like "I'd like the JetBlue flight" maps to actions (e. g., selecting that flight).

Instruction Following Reinforcement Learning (RL)

Black-box Adversarial Attacks with Limited Queries and Information

2 code implementations • ICML 2018 • Andrew Ilyas, Logan Engstrom, Anish Athalye, Jessy Lin

Current neural network-based classifiers are susceptible to adversarial examples even in the black-box setting, where the attacker only has query access to the model.

Query-Efficient Black-box Adversarial Examples (superceded)

1 code implementation • 19 Dec 2017 • Andrew Ilyas, Logan Engstrom, Anish Athalye, Jessy Lin

Second, we introduce a new algorithm to perform targeted adversarial attacks in the partial-information setting, where the attacker only has access to a limited number of target classes.

Adversarial Attack