Search Results for author: Dan Klein

Found 88 papers, 40 papers with code

Train Big, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers

no code implementations ICML 2020 Zhuohan Li, Eric Wallace, Sheng Shen, Kevin Lin, Kurt Keutzer, Dan Klein, Joseph Gonzalez

Since hardware resources are limited, the objective of training deep learning models is typically to maximize accuracy subject to the time and memory constraints of training and inference.

Machine Translation Quantization

Learning Space Partitions for Path Planning

1 code implementation19 Jun 2021 Kevin Yang, Tianjun Zhang, Chris Cummins, Brandon Cui, Benoit Steiner, Linnan Wang, Joseph E. Gonzalez, Dan Klein, Yuandong Tian

Path planning, the problem of efficiently discovering high-reward trajectories, often requires optimizing a high-dimensional and multimodal reward function.

An Improved Model for Voicing Silent Speech

1 code implementation3 Jun 2021 David Gaddy, Dan Klein

In this paper, we present an improved model for voicing silent speech, where audio is synthesized from facial electromyography (EMG) signals.

Electromyography (EMG)

Are Larger Pretrained Language Models Uniformly Better? Comparing Performance at the Instance Level

1 code implementation13 May 2021 Ruiqi Zhong, Dhruba Ghosh, Dan Klein, Jacob Steinhardt

We develop statistically rigorous methods to address this, and after accounting for pretraining and finetuning noise, we find that our BERT-Large is worse than BERT-Mini on at least 1-4% of instances across MNLI, SST-2, and QQP, compared to the overall accuracy improvement of 2-10%.

FUDGE: Controlled Text Generation With Future Discriminators

1 code implementation NAACL 2021 Kevin Yang, Dan Klein

We propose Future Discriminators for Generation (FUDGE), a flexible and modular method for controlled text generation.

Machine Translation Text Generation

Meta-tuning Language Models to Answer Prompts Better

no code implementations10 Apr 2021 Ruiqi Zhong, Kristy Lee, Zheng Zhang, Dan Klein

After meta-tuning, our model outperforms a same-sized QA model for most labels on unseen tasks, and we forecast that the performance would improve for even larger models.

Language Modelling Question Answering +1

Approximating How Single Head Attention Learns

1 code implementation13 Mar 2021 Charlie Snell, Ruiqi Zhong, Dan Klein, Jacob Steinhardt

To formalize, we define a model property, Knowledge to Translate Individual Words (KTIW) (e. g. knowing that `i` translates to `o`), and claim that it drives the learning of the attention.

Calibrate Before Use: Improving Few-Shot Performance of Language Models

2 code implementations19 Feb 2021 Tony Z. Zhao, Eric Wallace, Shi Feng, Dan Klein, Sameer Singh

We show that this type of few-shot learning can be unstable: the choice of prompt format, training examples, and even the order of the training examples can cause accuracy to vary from near chance to near state-of-the-art.

Few-Shot Learning

Modular Networks for Compositional Instruction Following

no code implementations NAACL 2021 Rodolfo Corona, Daniel Fried, Coline Devin, Dan Klein, Trevor Darrell

In our approach, subgoal modules each carry out natural language instructions for a specific subgoal type.

Constructing Taxonomies from Pretrained Language Models

no code implementations NAACL 2021 Catherine Chen, Kevin Lin, Dan Klein

The tree reconciliation module treats the task as a graph optimization problem and outputs the maximum spanning tree of this graph.

Unsupervised Parsing via Constituency Tests

no code implementations EMNLP 2020 Steven Cao, Nikita Kitaev, Dan Klein

We propose a method for unsupervised parsing based on the linguistic notion of a constituency test.

Semantic Evaluation for Text-to-SQL with Distilled Test Suites

2 code implementations EMNLP 2020 Ruiqi Zhong, Tao Yu, Dan Klein

We propose test suite accuracy to approximate semantic accuracy for Text-to-SQL models.

Text-To-Sql

Digital Voicing of Silent Speech

1 code implementation EMNLP 2020 David Gaddy, Dan Klein

In this paper, we consider the task of digitally voicing silent speech, where silently mouthed words are converted to audible speech based on electromyography (EMG) sensor measurements that capture muscle impulses.

Electromyography (EMG) Speech Synthesis

A Streaming Approach For Efficient Batched Beam Search

1 code implementation EMNLP 2020 Kevin Yang, Violet Yao, John DeNero, Dan Klein

We propose an efficient batching strategy for variable-length decoding on GPU architectures.

Machine Translation

Semantic Scaffolds for Pseudocode-to-Code Generation

1 code implementation ACL 2020 Ruiqi Zhong, Mitchell Stern, Dan Klein

We propose a method for program generation based on semantic scaffolds, lightweight structures representing the high-level semantic and syntactic composition of a program.

Code Generation

Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers

1 code implementation26 Feb 2020 Zhuohan Li, Eric Wallace, Sheng Shen, Kevin Lin, Kurt Keutzer, Dan Klein, Joseph E. Gonzalez

Since hardware resources are limited, the objective of training deep learning models is typically to maximize accuracy subject to the time and memory constraints of training and inference.

Machine Translation Quantization

Multilingual Alignment of Contextual Word Representations

no code implementations ICLR 2020 Steven Cao, Nikita Kitaev, Dan Klein

We propose procedures for evaluating and strengthening contextual embedding alignment and show that they are useful in analyzing and improving multilingual BERT.

Pre-Learning Environment Representations for Data-Efficient Neural Instruction Following

1 code implementation ACL 2019 David Gaddy, Dan Klein

We consider the problem of learning to map from natural language instructions to state transitions (actions) in a data-efficient manner.

Cross-Domain Generalization of Neural Constituency Parsers

1 code implementation ACL 2019 Daniel Fried, Nikita Kitaev, Dan Klein

Neural parsers obtain state-of-the-art results on benchmark treebanks for constituency parsing -- but to what degree do they generalize to other domains?

Constituency Parsing Domain Generalization

Are You Looking? Grounding to Multiple Modalities in Vision-and-Language Navigation

no code implementations ACL 2019 Ronghang Hu, Daniel Fried, Anna Rohrbach, Dan Klein, Trevor Darrell, Kate Saenko

The actual grounding can connect language to the environment through multiple modalities, e. g. "stop at the door" might ground into visual objects, while "turn right" might rely only on the geometric structure of a route.

Vision and Language Navigation

Tetra-Tagging: Word-Synchronous Parsing with Linear-Time Inference

1 code implementation ACL 2020 Nikita Kitaev, Dan Klein

We present a constituency parsing algorithm that, like a supertagger, works by assigning labels to each word in a sentence.

Constituency Parsing

Multilingual Constituency Parsing with Self-Attention and Pre-Training

2 code implementations ACL 2019 Nikita Kitaev, Steven Cao, Dan Klein

We show that constituency parsing benefits from unsupervised pre-training across a variety of languages and a range of pre-training conditions.

Constituency Parsing Unsupervised Pre-training

Policy Gradient as a Proxy for Dynamic Oracles in Constituency Parsing

no code implementations ACL 2018 Daniel Fried, Dan Klein

Dynamic oracles provide strong supervision for training constituency parsers with exploration, but must be custom defined for a given parser's transition system.

Constituency Parsing

Speaker-Follower Models for Vision-and-Language Navigation

1 code implementation NeurIPS 2018 Daniel Fried, Ronghang Hu, Volkan Cirik, Anna Rohrbach, Jacob Andreas, Louis-Philippe Morency, Taylor Berg-Kirkpatrick, Kate Saenko, Dan Klein, Trevor Darrell

We use this speaker model to (1) synthesize new instructions for data augmentation and to (2) implement pragmatic reasoning, which evaluates how well candidate action sequences explain an instruction.

Data Augmentation Vision and Language Navigation

Constituency Parsing with a Self-Attentive Encoder

2 code implementations ACL 2018 Nikita Kitaev, Dan Klein

We demonstrate that replacing an LSTM encoder with a self-attentive architecture can lead to improvements to a state-of-the-art discriminative constituency parser.

Constituency Parsing

What's Going On in Neural Constituency Parsers? An Analysis

1 code implementation NAACL 2018 David Gaddy, Mitchell Stern, Dan Klein

A number of differences have emerged between modern and classic approaches to constituency parsing in recent years, with structural components like grammars and feature-rich lexicons becoming less central while recurrent neural network representations rise in popularity.

Constituency Parsing

Unified Pragmatic Models for Generating and Following Instructions

1 code implementation NAACL 2018 Daniel Fried, Jacob Andreas, Dan Klein

We show that explicit pragmatic inference aids in correctly generating and following natural language instructions for complex, sequential tasks.

Text Generation

Learning with Latent Language

1 code implementation NAACL 2018 Jacob Andreas, Dan Klein, Sergey Levine

The named concepts and compositional operators present in natural language provide a rich source of information about the kinds of abstractions humans use to navigate the world.

Image Classification

Effective Inference for Generative Neural Parsing

no code implementations EMNLP 2017 Mitchell Stern, Daniel Fried, Dan Klein

Generative neural models have recently achieved state-of-the-art results for constituency parsing.

Constituency Parsing

Analogs of Linguistic Structure in Deep Representations

3 code implementations EMNLP 2017 Jacob Andreas, Dan Klein

We investigate the compositional structure of message vectors computed by a deep network trained on a communication game.

Parsing with Traces: An $O(n^4)$ Algorithm and a Structural Representation

1 code implementation13 Jul 2017 Jonathan K. Kummerfeld, Dan Klein

General treebank analyses are graph structured, but parsers are typically restricted to tree structures for efficiency and modeling reasons.

Constituency Parsing Missing Elements

A Minimal Span-Based Neural Constituency Parser

no code implementations ACL 2017 Mitchell Stern, Jacob Andreas, Dan Klein

In this work, we present a minimal neural model for constituency parsing based on independent scoring of labels and spans.

Constituency Parsing

Abstract Syntax Networks for Code Generation and Semantic Parsing

1 code implementation ACL 2017 Maxim Rabinovich, Mitchell Stern, Dan Klein

Tasks like code generation and semantic parsing require mapping unstructured (or partially structured) inputs to well-formed, executable outputs.

Code Generation Semantic Parsing

Fine-Grained Entity Typing with High-Multiplicity Assignments

no code implementations ACL 2017 Maxim Rabinovich, Dan Klein

As entity type systems become richer and more fine-grained, we expect the number of types assigned to a given entity to increase.

Entity Typing

Translating Neuralese

1 code implementation ACL 2017 Jacob Andreas, Anca Dragan, Dan Klein

Several approaches have recently been proposed for learning decentralized deep multiagent policies that coordinate via a differentiable communication channel.

Machine Translation

Parsing with Traces: An O(n4) Algorithm and a Structural Representation

no code implementations TACL 2017 Jonathan K. Kummerfeld, Dan Klein

General treebank analyses are graph structured, but parsers are typically restricted to tree structures for efficiency and modeling reasons.

Question Answering

Capturing Semantic Similarity for Entity Linking with Convolutional Neural Networks

1 code implementation NAACL 2016 Matthew Francis-Landau, Greg Durrett, Dan Klein

A key challenge in entity linking is making effective use of contextual information to disambiguate mentions that might refer to different entities in different contexts.

Entity Linking Semantic correspondence +2

Reasoning About Pragmatics with Neural Listeners and Speakers

1 code implementation EMNLP 2016 Jacob Andreas, Dan Klein

We present a model for pragmatically describing scenes, in which contrastive behavior results from a combination of inference-driven pragmatics and learned semantics.

Text Generation

Learning-Based Single-Document Summarization with Compression and Anaphoricity Constraints

no code implementations ACL 2016 Greg Durrett, Taylor Berg-Kirkpatrick, Dan Klein

We present a discriminative model for single-document summarization that integrally combines compression and anaphoricity constraints.

Document Summarization

Neural Module Networks

1 code implementation CVPR 2016 Jacob Andreas, Marcus Rohrbach, Trevor Darrell, Dan Klein

Visual question answering is fundamentally compositional in nature---a question like "where is the dog?"

Visual Question Answering

Alignment-based compositional semantics for instruction following

1 code implementation EMNLP 2015 Jacob Andreas, Dan Klein

This paper describes an alignment-based model for interpreting natural language instructions in context.

Neural CRF Parsing

no code implementations IJCNLP 2015 Greg Durrett, Dan Klein

This paper describes a parsing model that combines the exact dynamic programming of CRF parsing with the rich nonlinear featurization of neural net approaches.

On the accuracy of self-normalized log-linear models

no code implementations NeurIPS 2015 Jacob Andreas, Maxim Rabinovich, Dan Klein, Michael. I. Jordan

Calculation of the log-normalizer is a major computational obstacle in applications of log-linear models with large output spaces.

Generalization Bounds

Unsupervised Transcription of Piano Music

no code implementations NeurIPS 2014 Taylor Berg-Kirkpatrick, Jacob Andreas, Dan Klein

We present a new probabilistic model for transcribing piano music from audio to a symbolic form.

A Joint Model for Entity Analysis: Coreference, Typing, and Linking

no code implementations TACL 2014 Greg Durrett, Dan Klein

We present a joint model of three core tasks in the entity analysis stack: coreference resolution (within-document clustering), named entity recognition (coarse semantic typing), and entity linking (matching to Wikipedia entities).

Coreference Resolution Entity Linking +1

Learning Semantic Correspondences with Less Supervision

1 code implementation1 Aug 2009 Percy Liang, Michael Jordan, Dan Klein

A central problem in grounded language acquisition is learning the correspondences between a rich world state and a stream of text which references that world state.

Language Acquisition

A Probabilistic Approach to Language Change

no code implementations NeurIPS 2007 Alexandre Bouchard-Côté, Percy S. Liang, Dan Klein, Thomas L. Griffiths

We present a probabilistic approach to language change in which word forms are represented by phoneme sequences that undergo stochastic edits along the branches of a phylogenetic tree.

Cannot find the paper you are looking for? You can Submit a new open access paper.