Search Results for author: Dan Klein

Found 124 papers, 65 papers with code

Learned Incremental Representations for Parsing

1 code implementation ACL 2022 Nikita Kitaev, Thomas Lu, Dan Klein

We present an incremental syntactic representation that consists of assigning a single discrete label to each word in a sentence, where the label is predicted using strictly incremental processing of a prefix of the sentence, and the sequence of labels for a sentence fully determines a parse tree.

Sentence

Train Big, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers

no code implementations ICML 2020 Zhuohan Li, Eric Wallace, Sheng Shen, Kevin Lin, Kurt Keutzer, Dan Klein, Joseph Gonzalez

Since hardware resources are limited, the objective of training deep learning models is typically to maximize accuracy subject to the time and memory constraints of training and inference.

Machine Translation Quantization +1

What Evidence Do Language Models Find Convincing?

1 code implementation19 Feb 2024 Alexander Wan, Eric Wallace, Dan Klein

Retrieval-augmented language models are being increasingly tasked with subjective, contentious, and conflicting queries such as "is aspartame linked to cancer".

counterfactual Misinformation

Prompted Contextual Vectors for Spear-Phishing Detection

1 code implementation13 Feb 2024 Daniel Nahmias, Gal Engelberg, Dan Klein, Asaf Shabtai

Spear-phishing attacks present a significant security challenge, with large language models (LLMs) escalating the threat by generating convincing emails and facilitating target reconnaissance.

Document Classification

Comparative Multi-View Language Grounding

no code implementations12 Nov 2023 Chancharik Mitra, Abrar Anwar, Rodolfo Corona, Dan Klein, Trevor Darrell, Jesse Thomason

In this work, we consider the task of resolving object referents when given a comparative language description.

Object

Improving Pacing in Long-Form Story Planning

1 code implementation8 Nov 2023 Yichen Wang, Kevin Yang, Xiaoming Liu, Dan Klein

Existing LLM-based systems for writing long-form stories or story outlines frequently suffer from unnatural pacing, whether glossing over important events or over-elaborating on insignificant details, resulting in a jarring experience for the reader.

Incorporating Worker Perspectives into MTurk Annotation Practices for NLP

no code implementations6 Nov 2023 Olivia Huang, Eve Fleisig, Dan Klein

Current practices regarding data collection for natural language processing on Amazon Mechanical Turk (MTurk) often rely on a combination of studies on data quality and heuristics shared among NLP researchers.

Can Language Models Learn to Listen?

no code implementations ICCV 2023 Evonne Ng, Sanjay Subramanian, Dan Klein, Angjoo Kanazawa, Trevor Darrell, Shiry Ginosar

We present a framework for generating appropriate facial responses from a listener in dyadic social interactions based on the speaker's words.

Language Modelling Large Language Model

Learning to Model the World with Language

no code implementations31 Jul 2023 Jessy Lin, Yuqing Du, Olivia Watkins, Danijar Hafner, Pieter Abbeel, Dan Klein, Anca Dragan

To interact with humans in the world, agents need to understand the diverse types of language that people use, relate them to the visual world, and act based on them.

Future prediction General Knowledge +1

RLCD: Reinforcement Learning from Contrastive Distillation for Language Model Alignment

2 code implementations24 Jul 2023 Kevin Yang, Dan Klein, Asli Celikyilmaz, Nanyun Peng, Yuandong Tian

We propose Reinforcement Learning from Contrastive Distillation (RLCD), a method for aligning language models to follow principles expressed in natural language (e. g., to be more harmless) without using human feedback.

Language Modelling reinforcement-learning

PREADD: Prefix-Adaptive Decoding for Controlled Text Generation

1 code implementation6 Jul 2023 Jonathan Pei, Kevin Yang, Dan Klein

We propose Prefix-Adaptive Decoding (PREADD), a flexible method for controlled text generation.

Attribute Text Generation

Are Layout-Infused Language Models Robust to Layout Distribution Shifts? A Case Study with Scientific Documents

1 code implementation1 Jun 2023 Catherine Chen, Zejiang Shen, Dan Klein, Gabriel Stanovsky, Doug Downey, Kyle Lo

Recent work has shown that infusing layout features into language models (LMs) improves processing of visually-rich documents such as scientific papers.

Decomposing Complex Queries for Tip-of-the-tongue Retrieval

no code implementations24 May 2023 Kevin Lin, Kyle Lo, Joseph E. Gonzalez, Dan Klein

When re-finding items, users who forget or are uncertain about identifying details often rely on creative strategies for expressing their information needs -- complex queries that describe content elements (e. g., book characters or events), information beyond the document text (e. g., descriptions of book covers), or personal context (e. g., when they read a book).

Retrieval

Ghostbuster: Detecting Text Ghostwritten by Large Language Models

1 code implementation24 May 2023 Vivek Verma, Eve Fleisig, Nicholas Tomlin, Dan Klein

In conjunction with our model, we release three new datasets of human- and AI-generated text as detection benchmarks in the domains of student essays, creative writing, and news articles.

Centering the Margins: Outlier-Based Identification of Harmed Populations in Toxicity Detection

no code implementations24 May 2023 Vyoma Raman, Eve Fleisig, Dan Klein

We also find text and demographic outliers to be particularly susceptible to errors in the classification of severe toxicity and identity attacks.

Outlier Detection

Revisiting Entropy Rate Constancy in Text

no code implementations20 May 2023 Vivek Verma, Nicholas Tomlin, Dan Klein

The uniform information density (UID) hypothesis states that humans tend to distribute information roughly evenly across an utterance or discourse.

When the Majority is Wrong: Modeling Annotator Disagreement for Subjective Tasks

no code implementations11 May 2023 Eve Fleisig, Rediet Abebe, Dan Klein

Thus, a crucial problem in hate speech detection is determining whether a statement is offensive to the demographic group that it targets, when that group may constitute a small fraction of the annotator pool.

Hate Speech Detection

Poisoning Language Models During Instruction Tuning

1 code implementation1 May 2023 Alexander Wan, Eric Wallace, Sheng Shen, Dan Klein

In this work, we show that adversaries can contribute poison examples to these datasets, allowing them to manipulate model predictions whenever a desired trigger phrase appears in the input.

A Vision-free Baseline for Multimodal Grammar Induction

no code implementations20 Dec 2022 Boyi Li, Rodolfo Corona, Karttikeya Mangalam, Catherine Chen, Daniel Flaherty, Serge Belongie, Kilian Q. Weinberger, Jitendra Malik, Trevor Darrell, Dan Klein

Compared to image-aided grammar induction, LC-PCFG outperforms the prior state-of-the-art by 7. 9 Corpus-F1 points, with an 85% reduction in parameter count and 1. 7x faster training speed.

Constituency Parsing

DOC: Improving Long Story Coherence With Detailed Outline Control

1 code implementation20 Dec 2022 Kevin Yang, Dan Klein, Nanyun Peng, Yuandong Tian

In human evaluations of automatically generated stories, DOC substantially outperforms a strong Re3 baseline (Yang et al., 2022) on plot coherence (22. 5% absolute gain), outline relevance (28. 2%), and interestingness (20. 7%).

Discovering Latent Knowledge in Language Models Without Supervision

1 code implementation7 Dec 2022 Collin Burns, Haotian Ye, Dan Klein, Jacob Steinhardt

Existing techniques for training language models can be misaligned with the truth: if we train models with imitation learning, they may reproduce errors that humans make; if we train them to generate text that humans rate highly, they may output errors that human evaluators can't detect.

Imitation Learning Language Modelling +2

Neural Unsupervised Reconstruction of Protolanguage Word Forms

no code implementations16 Nov 2022 Andre He, Nicholas Tomlin, Dan Klein

We evaluate our performance on the task of reconstructing Latin from a dataset of cognates across five Romance languages, achieving a notable reduction in edit distance from the target word forms compared to previous methods.

Re3: Generating Longer Stories With Recursive Reprompting and Revision

1 code implementation13 Oct 2022 Kevin Yang, Yuandong Tian, Nanyun Peng, Dan Klein

We consider the problem of automatically generating longer stories of over two thousand words.

Language Modelling

Learning by Distilling Context

no code implementations30 Sep 2022 Charlie Snell, Dan Klein, Ruiqi Zhong

We show that context distillation is a general method to train language models, and it can effectively internalize 3 types of training signals.

Language Modelling Text-To-SQL

Non-Programmers Can Label Programs Indirectly via Active Examples: A Case Study with Text-to-SQL

1 code implementation25 May 2022 Ruiqi Zhong, Charlie Snell, Dan Klein, Jason Eisner

We introduce APEL, a framework in which non-programmers select among candidate programs generated by a seed semantic parser (e. g., Codex).

Bayesian Inference Text-To-SQL

Voxel-informed Language Grounding

2 code implementations ACL 2022 Rodolfo Corona, Shizhan Zhu, Dan Klein, Trevor Darrell

Natural language applied to natural 2D images describes a fundamentally 3D world.

Automated Crossword Solving

1 code implementation ACL 2022 Eric Wallace, Nicholas Tomlin, Albert Xu, Kevin Yang, Eshaan Pathak, Matthew Ginsberg, Dan Klein

We present the Berkeley Crossword Solver, a state-of-the-art approach for automatically solving crossword puzzles.

Question Answering

Understanding Game-Playing Agents with Natural Language Annotations

1 code implementation ACL 2022 Nicholas Tomlin, Andre He, Dan Klein

We present a new dataset containing 10K human-annotated games of Go and show how these natural language annotations can be used as a tool for model interpretability.

Imitation Learning Reinforcement Learning (RL)

Inferring Rewards from Language in Context

1 code implementation ACL 2022 Jessy Lin, Daniel Fried, Dan Klein, Anca Dragan

In classic instruction following, language like "I'd like the JetBlue flight" maps to actions (e. g., selecting that flight).

Instruction Following Reinforcement Learning (RL)

Describing Differences between Text Distributions with Natural Language

1 code implementation28 Jan 2022 Ruiqi Zhong, Charlie Snell, Dan Klein, Jacob Steinhardt

We then re-rank the descriptions by checking how often they hold on a larger set of samples with a learned verifier.

Binary Classification Re-Ranking

Reference-Centric Models for Grounded Collaborative Dialogue

1 code implementation EMNLP 2021 Daniel Fried, Justin T. Chiu, Dan Klein

We present a grounded neural dialogue model that successfully collaborates with people in a partially-observable reference game.

Value-Agnostic Conversational Semantic Parsing

no code implementations ACL 2021 Emmanouil Antonios Platanios, Adam Pauls, Subhro Roy, Yuchen Zhang, Alexander Kyte, Alan Guo, Sam Thomson, Jayant Krishnamurthy, Jason Wolfe, Jacob Andreas, Dan Klein

Conversational semantic parsers map user utterances to executable programs given dialogue histories composed of previous utterances, programs, and system responses.

Computational Efficiency Semantic Parsing

Learning Space Partitions for Path Planning

2 code implementations NeurIPS 2021 Kevin Yang, Tianjun Zhang, Chris Cummins, Brandon Cui, Benoit Steiner, Linnan Wang, Joseph E. Gonzalez, Dan Klein, Yuandong Tian

Path planning, the problem of efficiently discovering high-reward trajectories, often requires optimizing a high-dimensional and multimodal reward function.

An Improved Model for Voicing Silent Speech

1 code implementation ACL 2021 David Gaddy, Dan Klein

In this paper, we present an improved model for voicing silent speech, where audio is synthesized from facial electromyography (EMG) signals.

Electromyography (EMG)

Are Larger Pretrained Language Models Uniformly Better? Comparing Performance at the Instance Level

1 code implementation Findings (ACL) 2021 Ruiqi Zhong, Dhruba Ghosh, Dan Klein, Jacob Steinhardt

We develop statistically rigorous methods to address this, and after accounting for pretraining and finetuning noise, we find that our BERT-Large is worse than BERT-Mini on at least 1-4% of instances across MNLI, SST-2, and QQP, compared to the overall accuracy improvement of 2-10%.

QQP SST-2

FUDGE: Controlled Text Generation With Future Discriminators

3 code implementations NAACL 2021 Kevin Yang, Dan Klein

We propose Future Discriminators for Generation (FUDGE), a flexible and modular method for controlled text generation.

Attribute Machine Translation +2

Approximating How Single Head Attention Learns

1 code implementation13 Mar 2021 Charlie Snell, Ruiqi Zhong, Dan Klein, Jacob Steinhardt

Our approximation explains why models sometimes attend to salient words, and inspires a toy example where a multi-head attention model can overcome the above hard training distribution by improving learning dynamics rather than expressiveness.

Calibrate Before Use: Improving Few-Shot Performance of Language Models

5 code implementations19 Feb 2021 Tony Z. Zhao, Eric Wallace, Shi Feng, Dan Klein, Sameer Singh

We show that this type of few-shot learning can be unstable: the choice of prompt format, training examples, and even the order of the training examples can cause accuracy to vary from near chance to near state-of-the-art.

Few-Shot Learning

Constructing Taxonomies from Pretrained Language Models

no code implementations NAACL 2021 Catherine Chen, Kevin Lin, Dan Klein

The tree reconciliation module treats the task as a graph optimization problem and outputs the maximum spanning tree of this graph.

Unsupervised Parsing via Constituency Tests

no code implementations EMNLP 2020 Steven Cao, Nikita Kitaev, Dan Klein

We propose a method for unsupervised parsing based on the linguistic notion of a constituency test.

Sentence

Semantic Evaluation for Text-to-SQL with Distilled Test Suites

3 code implementations EMNLP 2020 Ruiqi Zhong, Tao Yu, Dan Klein

We propose test suite accuracy to approximate semantic accuracy for Text-to-SQL models.

Text-To-SQL

Digital Voicing of Silent Speech

1 code implementation EMNLP 2020 David Gaddy, Dan Klein

In this paper, we consider the task of digitally voicing silent speech, where silently mouthed words are converted to audible speech based on electromyography (EMG) sensor measurements that capture muscle impulses.

Electromyography (EMG) Speech Synthesis

Understanding Attention Training via Output Relevance

no code implementations16 Aug 2020 Charlie Snell, Ruiqi Zhong, Jacob Steinhardt, Dan Klein

If we ablate attention by fixing it to uniform, the output relevance still correlates with the attention of a normally trained model; but if we instead ablate output relevance, attention cannot be learned.

Translation

Semantic Evaluation for Text-to-SQL with Distilled Test Suite

no code implementations2 Jul 2020 Ruiqi Zhong, Tao Yu, Dan Klein

We propose test suite accuracy to approximate semantic accuracy for Text-to-SQL models, where a predicted query is semantically correct if its denotation is the same as the gold for every possible database.

Semantic Parsing Text-To-SQL

Semantic Scaffolds for Pseudocode-to-Code Generation

1 code implementation ACL 2020 Ruiqi Zhong, Mitchell Stern, Dan Klein

We propose a method for program generation based on semantic scaffolds, lightweight structures representing the high-level semantic and syntactic composition of a program.

Code Generation

Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers

2 code implementations26 Feb 2020 Zhuohan Li, Eric Wallace, Sheng Shen, Kevin Lin, Kurt Keutzer, Dan Klein, Joseph E. Gonzalez

Since hardware resources are limited, the objective of training deep learning models is typically to maximize accuracy subject to the time and memory constraints of training and inference.

Machine Translation Quantization +1

Multilingual Alignment of Contextual Word Representations

no code implementations ICLR 2020 Steven Cao, Nikita Kitaev, Dan Klein

We propose procedures for evaluating and strengthening contextual embedding alignment and show that they are useful in analyzing and improving multilingual BERT.

Retrieval

Pre-Learning Environment Representations for Data-Efficient Neural Instruction Following

1 code implementation ACL 2019 David Gaddy, Dan Klein

We consider the problem of learning to map from natural language instructions to state transitions (actions) in a data-efficient manner.

Instruction Following

Cross-Domain Generalization of Neural Constituency Parsers

1 code implementation ACL 2019 Daniel Fried, Nikita Kitaev, Dan Klein

Neural parsers obtain state-of-the-art results on benchmark treebanks for constituency parsing -- but to what degree do they generalize to other domains?

Constituency Parsing Domain Generalization

Are You Looking? Grounding to Multiple Modalities in Vision-and-Language Navigation

no code implementations ACL 2019 Ronghang Hu, Daniel Fried, Anna Rohrbach, Dan Klein, Trevor Darrell, Kate Saenko

The actual grounding can connect language to the environment through multiple modalities, e. g. "stop at the door" might ground into visual objects, while "turn right" might rely only on the geometric structure of a route.

Vision and Language Navigation

Tetra-Tagging: Word-Synchronous Parsing with Linear-Time Inference

2 code implementations ACL 2020 Nikita Kitaev, Dan Klein

We present a constituency parsing algorithm that, like a supertagger, works by assigning labels to each word in a sentence.

Constituency Parsing Sentence

Multilingual Constituency Parsing with Self-Attention and Pre-Training

4 code implementations ACL 2019 Nikita Kitaev, Steven Cao, Dan Klein

We show that constituency parsing benefits from unsupervised pre-training across a variety of languages and a range of pre-training conditions.

Constituency Parsing Unsupervised Pre-training

Policy Gradient as a Proxy for Dynamic Oracles in Constituency Parsing

no code implementations ACL 2018 Daniel Fried, Dan Klein

Dynamic oracles provide strong supervision for training constituency parsers with exploration, but must be custom defined for a given parser's transition system.

Constituency Parsing

Speaker-Follower Models for Vision-and-Language Navigation

1 code implementation NeurIPS 2018 Daniel Fried, Ronghang Hu, Volkan Cirik, Anna Rohrbach, Jacob Andreas, Louis-Philippe Morency, Taylor Berg-Kirkpatrick, Kate Saenko, Dan Klein, Trevor Darrell

We use this speaker model to (1) synthesize new instructions for data augmentation and to (2) implement pragmatic reasoning, which evaluates how well candidate action sequences explain an instruction.

Data Augmentation Vision and Language Navigation

Constituency Parsing with a Self-Attentive Encoder

4 code implementations ACL 2018 Nikita Kitaev, Dan Klein

We demonstrate that replacing an LSTM encoder with a self-attentive architecture can lead to improvements to a state-of-the-art discriminative constituency parser.

Constituency Parsing Sentence

What's Going On in Neural Constituency Parsers? An Analysis

1 code implementation NAACL 2018 David Gaddy, Mitchell Stern, Dan Klein

A number of differences have emerged between modern and classic approaches to constituency parsing in recent years, with structural components like grammars and feature-rich lexicons becoming less central while recurrent neural network representations rise in popularity.

Constituency Parsing

Unified Pragmatic Models for Generating and Following Instructions

1 code implementation NAACL 2018 Daniel Fried, Jacob Andreas, Dan Klein

We show that explicit pragmatic inference aids in correctly generating and following natural language instructions for complex, sequential tasks.

Text Generation

Learning with Latent Language

1 code implementation NAACL 2018 Jacob Andreas, Dan Klein, Sergey Levine

The named concepts and compositional operators present in natural language provide a rich source of information about the kinds of abstractions humans use to navigate the world.

Image Classification Navigate

Effective Inference for Generative Neural Parsing

no code implementations EMNLP 2017 Mitchell Stern, Daniel Fried, Dan Klein

Generative neural models have recently achieved state-of-the-art results for constituency parsing.

Constituency Parsing

Analogs of Linguistic Structure in Deep Representations

3 code implementations EMNLP 2017 Jacob Andreas, Dan Klein

We investigate the compositional structure of message vectors computed by a deep network trained on a communication game.

Negation

Parsing with Traces: An $O(n^4)$ Algorithm and a Structural Representation

1 code implementation13 Jul 2017 Jonathan K. Kummerfeld, Dan Klein

General treebank analyses are graph structured, but parsers are typically restricted to tree structures for efficiency and modeling reasons.

Constituency Parsing Missing Elements

A Minimal Span-Based Neural Constituency Parser

no code implementations ACL 2017 Mitchell Stern, Jacob Andreas, Dan Klein

In this work, we present a minimal neural model for constituency parsing based on independent scoring of labels and spans.

Constituency Parsing

Abstract Syntax Networks for Code Generation and Semantic Parsing

1 code implementation ACL 2017 Maxim Rabinovich, Mitchell Stern, Dan Klein

Tasks like code generation and semantic parsing require mapping unstructured (or partially structured) inputs to well-formed, executable outputs.

Code Generation Semantic Parsing

Fine-Grained Entity Typing with High-Multiplicity Assignments

no code implementations ACL 2017 Maxim Rabinovich, Dan Klein

As entity type systems become richer and more fine-grained, we expect the number of types assigned to a given entity to increase.

Entity Typing Vocal Bursts Intensity Prediction +1

Translating Neuralese

1 code implementation ACL 2017 Jacob Andreas, Anca Dragan, Dan Klein

Several approaches have recently been proposed for learning decentralized deep multiagent policies that coordinate via a differentiable communication channel.

Machine Translation Translation

Parsing with Traces: An O(n4) Algorithm and a Structural Representation

no code implementations TACL 2017 Jonathan K. Kummerfeld, Dan Klein

General treebank analyses are graph structured, but parsers are typically restricted to tree structures for efficiency and modeling reasons.

Question Answering

Capturing Semantic Similarity for Entity Linking with Convolutional Neural Networks

1 code implementation NAACL 2016 Matthew Francis-Landau, Greg Durrett, Dan Klein

A key challenge in entity linking is making effective use of contextual information to disambiguate mentions that might refer to different entities in different contexts.

Entity Linking Semantic correspondence +2

Reasoning About Pragmatics with Neural Listeners and Speakers

1 code implementation EMNLP 2016 Jacob Andreas, Dan Klein

We present a model for pragmatically describing scenes, in which contrastive behavior results from a combination of inference-driven pragmatics and learned semantics.

Referring Expression Text Generation

Learning-Based Single-Document Summarization with Compression and Anaphoricity Constraints

no code implementations ACL 2016 Greg Durrett, Taylor Berg-Kirkpatrick, Dan Klein

We present a discriminative model for single-document summarization that integrally combines compression and anaphoricity constraints.

Document Summarization Sentence

Neural Module Networks

1 code implementation CVPR 2016 Jacob Andreas, Marcus Rohrbach, Trevor Darrell, Dan Klein

Visual question answering is fundamentally compositional in nature---a question like "where is the dog?"

Visual Question Answering

Alignment-based compositional semantics for instruction following

1 code implementation EMNLP 2015 Jacob Andreas, Dan Klein

This paper describes an alignment-based model for interpreting natural language instructions in context.

Instruction Following Sentence

Neural CRF Parsing

no code implementations IJCNLP 2015 Greg Durrett, Dan Klein

This paper describes a parsing model that combines the exact dynamic programming of CRF parsing with the rich nonlinear featurization of neural net approaches.

On the accuracy of self-normalized log-linear models

no code implementations NeurIPS 2015 Jacob Andreas, Maxim Rabinovich, Dan Klein, Michael. I. Jordan

Calculation of the log-normalizer is a major computational obstacle in applications of log-linear models with large output spaces.

Generalization Bounds

Unsupervised Transcription of Piano Music

no code implementations NeurIPS 2014 Taylor Berg-Kirkpatrick, Jacob Andreas, Dan Klein

We present a new probabilistic model for transcribing piano music from audio to a symbolic form.

A Joint Model for Entity Analysis: Coreference, Typing, and Linking

no code implementations TACL 2014 Greg Durrett, Dan Klein

We present a joint model of three core tasks in the entity analysis stack: coreference resolution (within-document clustering), named entity recognition (coarse semantic typing), and entity linking (matching to Wikipedia entities).

Clustering coreference-resolution +4

Learning Semantic Correspondences with Less Supervision

1 code implementation1 Aug 2009 Percy Liang, Michael Jordan, Dan Klein

A central problem in grounded language acquisition is learning the correspondences between a rich world state and a stream of text which references that world state.

Language Acquisition

A Probabilistic Approach to Language Change

no code implementations NeurIPS 2007 Alexandre Bouchard-Côté, Percy S. Liang, Dan Klein, Thomas L. Griffiths

We present a probabilistic approach to language change in which word forms are represented by phoneme sequences that undergo stochastic edits along the branches of a phylogenetic tree.

Cannot find the paper you are looking for? You can Submit a new open access paper.