Search Results for author:
Found 88 papers, 40 papers with code
Train Big, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers
Since hardware resources are limited, the objective of training deep learning models is typically to maximize accuracy subject to the time and memory constraints of training and inference.
Learning Space Partitions for Path Planning
Path planning, the problem of efficiently discovering high-reward trajectories, often requires optimizing a high-dimensional and multimodal reward function.
An Improved Model for Voicing Silent Speech
In this paper, we present an improved model for voicing silent speech, where audio is synthesized from facial electromyography (EMG) signals.
Are Larger Pretrained Language Models Uniformly Better? Comparing Performance at the Instance Level
We develop statistically rigorous methods to address this, and after accounting for pretraining and finetuning noise, we find that our BERT-Large is worse than BERT-Mini on at least 1-4% of instances across MNLI, SST-2, and QQP, compared to the overall accuracy improvement of 2-10%.
Constrained Language Models Yield Few-Shot Semantic Parsers
We explore the use of large pretrained language models as few-shot semantic parsers.
Detoxifying Language Models Risks Marginalizing Minority Voices
Language models (LMs) must be both safe and equitable to be responsibly deployed in practice.
FUDGE: Controlled Text Generation With Future Discriminators
We propose Future Discriminators for Generation (FUDGE), a flexible and modular method for controlled text generation.
Meta-tuning Language Models to Answer Prompts Better
After meta-tuning, our model outperforms a same-sized QA model for most labels on unseen tasks, and we forecast that the performance would improve for even larger models.
Approximating How Single Head Attention Learns
To formalize, we define a model property, Knowledge to Translate Individual Words (KTIW) (e. g. knowing that `i` translates to `o`), and claim that it drives the learning of the attention.
Calibrate Before Use: Improving Few-Shot Performance of Language Models
We show that this type of few-shot learning can be unstable: the choice of prompt format, training examples, and even the order of the training examples can cause accuracy to vary from near chance to near state-of-the-art.
Modular Networks for Compositional Instruction Following
In our approach, subgoal modules each carry out natural language instructions for a specific subgoal type.
Constructing Taxonomies from Pretrained Language Models
The tree reconciliation module treats the task as a graph optimization problem and outputs the maximum spanning tree of this graph.
Unsupervised Parsing via Constituency Tests
We propose a method for unsupervised parsing based on the linguistic notion of a constituency test.
Semantic Evaluation for Text-to-SQL with Distilled Test Suites
We propose test suite accuracy to approximate semantic accuracy for Text-to-SQL models.
Digital Voicing of Silent Speech
In this paper, we consider the task of digitally voicing silent speech, where silently mouthed words are converted to audible speech based on electromyography (EMG) sensor measurements that capture muscle impulses.
A Streaming Approach For Efficient Batched Beam Search
We propose an efficient batching strategy for variable-length decoding on GPU architectures.
Task-Oriented Dialogue as Dataflow Synthesis
We describe an approach to task-oriented dialogue in which dialogue state is represented as a dataflow graph.
Semantic Scaffolds for Pseudocode-to-Code Generation
We propose a method for program generation based on semantic scaffolds, lightweight structures representing the high-level semantic and syntactic composition of a program.
Train Large, Then Compress: Rethinking Model Size for Efficient Training and Inference of Transformers
Since hardware resources are limited, the objective of training deep learning models is typically to maximize accuracy subject to the time and memory constraints of training and inference.
Multilingual Alignment of Contextual Word Representations
We propose procedures for evaluating and strengthening contextual embedding alignment and show that they are useful in analyzing and improving multilingual BERT.
A Deep Factorization of Style and Structure in Fonts
We propose a deep factorization model for typographic analysis that disentangles content from style.
Pre-Learning Environment Representations for Data-Efficient Neural Instruction Following
We consider the problem of learning to map from natural language instructions to state transitions (actions) in a data-efficient manner.
Cross-Domain Generalization of Neural Constituency Parsers
Neural parsers obtain state-of-the-art results on benchmark treebanks for constituency parsing -- but to what degree do they generalize to other domains?
Are You Looking? Grounding to Multiple Modalities in Vision-and-Language Navigation
The actual grounding can connect language to the environment through multiple modalities, e. g. "stop at the door" might ground into visual objects, while "turn right" might rely only on the geometric structure of a route.
Tetra-Tagging: Word-Synchronous Parsing with Linear-Time Inference
We present a constituency parsing algorithm that, like a supertagger, works by assigning labels to each word in a sentence.
Pragmatically Informative Text Generation
We improve the informativeness of models for conditional text generation using techniques from computational pragmatics.
Ranked #1 on
Data-to-Text Generation
on E2E NLG Challenge
Abstractive Text Summarization
Conditional Text Generation
+2
Multilingual Constituency Parsing with Self-Attention and Pre-Training
We show that constituency parsing benefits from unsupervised pre-training across a variety of languages and a range of pre-training conditions.
Ranked #3 on
Constituency Parsing
on CTB5
Policy Gradient as a Proxy for Dynamic Oracles in Constituency Parsing
Dynamic oracles provide strong supervision for training constituency parsers with exploration, but must be custom defined for a given parser's transition system.
Speaker-Follower Models for Vision-and-Language Navigation
We use this speaker model to (1) synthesize new instructions for data augmentation and to (2) implement pragmatic reasoning, which evaluates how well candidate action sequences explain an instruction.
Constituency Parsing with a Self-Attentive Encoder
We demonstrate that replacing an LSTM encoder with a self-attentive architecture can lead to improvements to a state-of-the-art discriminative constituency parser.
Ranked #6 on
Constituency Parsing
on CTB5
What's Going On in Neural Constituency Parsers? An Analysis
A number of differences have emerged between modern and classic approaches to constituency parsing in recent years, with structural components like grammars and feature-rich lexicons becoming less central while recurrent neural network representations rise in popularity.
Unified Pragmatic Models for Generating and Following Instructions
We show that explicit pragmatic inference aids in correctly generating and following natural language instructions for complex, sequential tasks.
Learning with Latent Language
The named concepts and compositional operators present in natural language provide a rich source of information about the kinds of abstractions humans use to navigate the world.
Where is Misty? Interpreting Spatial Descriptors by Modeling Regions in Space
We present a model for locating regions in space based on natural language descriptions.
Effective Inference for Generative Neural Parsing
Generative neural models have recently achieved state-of-the-art results for constituency parsing.
Analogs of Linguistic Structure in Deep Representations
We investigate the compositional structure of message vectors computed by a deep network trained on a communication game.
Parsing with Traces: An $O(n^4)$ Algorithm and a Structural Representation
General treebank analyses are graph structured, but parsers are typically restricted to tree structures for efficiency and modeling reasons.
Ranked #2 on
Missing Elements
on Penn Treebank
Improving Neural Parsing by Disentangling Model Combination and Reranking Effects
Recent work has proposed several generative neural models for constituency parsing that achieve state-of-the-art results.
Ranked #11 on
Constituency Parsing
on Penn Treebank
A Minimal Span-Based Neural Constituency Parser
In this work, we present a minimal neural model for constituency parsing based on independent scoring of labels and spans.
Abstract Syntax Networks for Code Generation and Semantic Parsing
Tasks like code generation and semantic parsing require mapping unstructured (or partially structured) inputs to well-formed, executable outputs.
Ranked #3 on
Semantic Parsing
on ATIS
Fine-Grained Entity Typing with High-Multiplicity Assignments
As entity type systems become richer and more fine-grained, we expect the number of types assigned to a given entity to increase.
Translating Neuralese
Several approaches have recently been proposed for learning decentralized deep multiagent policies that coordinate via a differentiable communication channel.
Parsing with Traces: An O(n4) Algorithm and a Structural Representation
General treebank analyses are graph structured, but parsers are typically restricted to tree structures for efficiency and modeling reasons.
Modular Multitask Reinforcement Learning with Policy Sketches
We describe a framework for multitask deep reinforcement learning guided by policy sketches.
Capturing Semantic Similarity for Entity Linking with Convolutional Neural Networks
A key challenge in entity linking is making effective use of contextual information to disambiguate mentions that might refer to different entities in different contexts.
Reasoning About Pragmatics with Neural Listeners and Speakers
We present a model for pragmatically describing scenes, in which contrastive behavior results from a combination of inference-driven pragmatics and learned semantics.
Learning-Based Single-Document Summarization with Compression and Anaphoricity Constraints
We present a discriminative model for single-document summarization that integrally combines compression and anaphoricity constraints.
Learning to Compose Neural Networks for Question Answering
We describe a question answering model that applies to both images and structured knowledge bases.
Neural Module Networks
Visual question answering is fundamentally compositional in nature---a question like "where is the dog?"
Ranked #6 on
Visual Question Answering
on VQA v1 test-std
Alignment-based compositional semantics for instruction following
This paper describes an alignment-based model for interpreting natural language instructions in context.
Neural CRF Parsing
This paper describes a parsing model that combines the exact dynamic programming of CRF parsing with the rich nonlinear featurization of neural net approaches.
On the accuracy of self-normalized log-linear models
Calculation of the log-normalizer is a major computational obstacle in applications of log-linear models with large output spaces.
Unsupervised Transcription of Piano Music
We present a new probabilistic model for transcribing piano music from audio to a symbolic form.
A Joint Model for Entity Analysis: Coreference, Typing, and Linking
We present a joint model of three core tasks in the entity analysis stack: coreference resolution (within-document clustering), named entity recognition (coarse semantic typing), and entity linking (matching to Wikipedia entities).
Ranked #19 on
Named Entity Recognition
on Ontonotes v5 (English)
Randomized Pruning: Efficiently Calculating Expectations in Large Dynamic Programs
Pruning can massively accelerate the computation of feature expectations in large models.
Learning Semantic Correspondences with Less Supervision
A central problem in grounded language acquisition is learning the correspondences between a rich world state and a stream of text which references that world state.
Efficient Inference in Phylogenetic InDel Trees
Accurate and efficient inference in evolutionary trees is a central problem in computational biology.
A Probabilistic Approach to Language Change
We present a probabilistic approach to language change in which word forms are represented by phoneme sequences that undergo stochastic edits along the branches of a phylogenetic tree.
Corpus-Based Induction of Syntactic Structure: Models of Dependency and Constituency
We present a generative model for the unsupervised learning of dependency structures.
Ranked #5 on
Unsupervised Dependency Parsing
on Penn Treebank