Search Results for author:
Found 71 papers, 58 papers with code
PromptSource: An Integrated Development Environment and Repository for Natural Language Prompts
PromptSource is a system for creating, sharing, and using natural language prompts.
Low-Rank Constraints for Fast Inference in Structured Models
This work demonstrates a simple approach to reduce the computational and memory complexity of a large class of structured models.
GenNI: Human-AI Collaboration for Data-Backed Text Generation
These controls allow users to globally constrain model generations, without sacrificing the representation power of the deep learning models.
Multitask Prompted Training Enables Zero-Shot Task Generalization
Large language models have recently been shown to attain reasonable zero-shot generalization on a diverse set of tasks (Brown et al., 2020).
Rationales for Sequential Predictions
Compared to existing baselines, greedy rationalization is best at optimizing the combinatorial objective and provides the most faithful rationales.
Block Pruning For Faster Transformers
Pre-training has improved model accuracy for both classification and generation tasks at the cost of introducing much larger and slower models.
Datasets: A Community Library for Natural Language Processing
The scale, variety, and quantity of publicly-available NLP datasets has grown rapidly as researchers propose new tasks, larger models, and novel benchmarks.
Low-Complexity Probing via Finding Subnetworks
The dominant approach in probing neural networks for linguistic properties is to train a new shallow multi-layer perceptron (MLP) on top of the model's internal representations.
How Many Data Points is a Prompt Worth?
When fine-tuning pretrained models for classification, researchers either use a generic model head or a task-specific prompt for prediction.
Named Tensor Notation
We propose a notation for tensors with named axes, which relieves the author, reader, and future implementers from the burden of keeping track of the order of axes and the purpose of each.
Parameter-Efficient Transfer Learning with Diff Pruning
This approach views finetuning as learning a task-specific diff vector that is applied on top of the pretrained parameter vector, which remains fixed and is shared across different tasks.
Learning from others' mistakes: Avoiding dataset biases without modeling them
State-of-the-art natural language processing (NLP) models often learn to model dataset biases and surface form correlations instead of features that target the intended underlying task.
Latent Template Induction with Gumbel-CRFs
Learning to control the structure of sentences is a challenging problem in text generation.
EdgeBERT: Sentence-Level Energy Optimizations for Latency-Aware Multi-Task NLP Inference
Transformer-based language models such as BERT provide significant accuracy improvement for a multitude of natural language processing (NLP) tasks.
Sequence-Level Mixed Sample Data Augmentation
Despite their empirical success, neural networks still have difficulty capturing compositional aspects of natural language.
Scaling Hidden Markov Language Models
The hidden Markov model (HMM) is a fundamental tool for sequence modeling that cleanly separates the hidden state from the emission structure.
Adversarial Semantic Collisions
We study semantic collisions: texts that are semantically unrelated but judged as similar by NLP models.
Pre-trained Summarization Distillation
A third, simpler approach is to 'shrink and fine-tune' (SFT), which avoids any explicit distillation by copying parameters to a smaller student model and then fine-tuning.
GRIT: Generative Role-filler Transformers for Document-level Event Entity Extraction
We revisit the classic problem of document-level role-filler entity extraction (REE) for template filling.
Cascaded Text Generation with Markov Transformers
The two dominant approaches to neural text generation are fully autoregressive models, using serial beam search decoding, and non-autoregressive models, using parallel decoding with no output dependencies.
Movement Pruning: Adaptive Sparsity by Fine-Tuning
Magnitude pruning is a widely used strategy for reducing model size in pure supervised learning; however, it is less effective in the transfer learning regime that has become standard for state-of-the-art natural language processing applications.
Posterior Control of Blackbox Generation
In this work, we consider augmenting neural generation models with discrete control states learned through a structured latent-variable approach.
What is Learned in Visually Grounded Neural Syntax Acquisition
Visual features are a promising signal for learning bootstrap textual models.
Automating Botnet Detection with Graph Neural Networks
Botnets are now a major source for many network attacks, such as DDoS attacks and spam.
Torch-Struct: Deep Structured Prediction Library
The literature on structured prediction for NLP describes a rich collection of distributions and algorithms over sequences, segmentations, alignments, and trees; however, these algorithms are difficult to utilize in deep learning frameworks.
A Hierarchy of Graph Neural Networks Based on Learnable Local Features
We derive theoretical results about the discriminative power and feature representation capabilities of each class.
HuggingFace's Transformers: State-of-the-art Natural Language Processing
Transformer architectures have facilitated building higher-capacity models and pretraining has made it possible to effectively utilize this capacity for a wide variety of tasks.
LAN -- A materials notation for 2D layered assemblies
Two-dimensional (2D) layered materials offer intriguing possibilities for novel physics and applications.
Materials Science
Neural Linguistic Steganography
Whereas traditional cryptography encrypts a secret message into an unintelligible form, steganography conceals that communication is taking place by encoding a secret message into a cover signal.
Commonsense Knowledge Mining from Pretrained Models
Inferring commonsense knowledge is a key challenge in natural language processing, but due to the sparsity of training data, previous work has shown that supervised methods for commonsense knowledge mining underperform when evaluated on novel data.
MASR: A Modular Accelerator for Sparse RNNs
The architecture is enhanced by a series of dynamic activation optimizations that enable compact storage, ensure no energy is wasted computing null operations, and maintain high MAC utilization for highly parallel accelerator designs.
Encoder-Agnostic Adaptation for Conditional Language Generation
Large pretrained language models have changed the way researchers approach discriminative natural language understanding tasks, leading to the dominance of approaches that adapt a pretrained model for arbitrary downstream tasks.
Simple Unsupervised Summarization by Contextual Matching
We propose an unsupervised method for sentence summarization using only language modeling.
Ranked #35 on
Text Summarization
on GigaWord
Visual Interaction with Deep Learning Models through Collaborative Semantic Inference
Automation of tasks can have critical consequences when humans lose agency over decision processes.
Don't Take the Premise for Granted: Mitigating Artifacts in Natural Language Inference
In contrast to standard approaches to NLI, our methods predict the probability of a premise given a hypothesis and NLI label, discouraging models from ignoring the premise.
On Adversarial Removal of Hypothesis-only Bias in Natural Language Inference
Popular Natural Language Inference (NLI) datasets have been shown to be tainted by hypothesis-only biases.
Compound Probabilistic Context-Free Grammars for Grammar Induction
We study a formalization of the grammar induction problem that models sentences as being generated by a compound probabilistic context-free grammar.
Ranked #5 on
Constituency Grammar Induction
on PTB
GLTR: Statistical Detection and Visualization of Generated Text
The rapid improvement of language models has raised the specter of abuse of text generation systems.
Unsupervised Recurrent Neural Network Grammars
On language modeling, unsupervised RNNGs perform as well their supervised counterparts on benchmarks in English and Chinese.
Ranked #6 on
Constituency Grammar Induction
on PTB
(Max F1 (WSJ) metric)
Latent Normalizing Flows for Discrete Sequences
Normalizing flows are a powerful class of generative models for continuous random variables, showing both strong model flexibility and the potential for non-autoregressive generation.
A Tutorial on Deep Latent Variable Models of Natural Language
There has been much recent, exciting work on combining the complementary strengths of latent variable models and deep learning.
End-to-End Content and Plan Selection for Data-to-Text Generation
Learning to generate fluent natural language from structured data with neural networks has become an common approach for NLG.
Entity Tracking Improves Cloze-style Reading Comprehension
Reading comprehension tasks test the ability of models to process long-term context and remember salient information.
Bottom-Up Abstractive Summarization
We use this selector as a bottom-up attention step to constrain the model to likely phrases.
Ranked #4 on
Multi-Document Summarization
on Multi-News
Learning Neural Templates for Text Generation
While neural, encoder-decoder models have had significant empirical success in text generation, there remain several unaddressed problems with this style of generation.
Avoiding Latent Variable Collapse With Generative Skip Models
VAEs can capture complex distributions, but they can also suffer from an issue known as "latent variable collapse," especially if the likelihood model is powerful.
Latent Alignment and Variational Attention
This work considers variational attention networks, alternatives to soft and hard attention for learning latent variable alignment models, with tighter approximation bounds based on amortized variational inference.
Ranked #23 on
Machine Translation
on IWSLT2014 German-English
OpenNMT: Neural Machine Translation Toolkit
OpenNMT is an open-source toolkit for neural machine translation (NMT).
Seq2Seq-Vis: A Visual Debugging Tool for Sequence-to-Sequence Models
In this work, we present a visual analysis tool that allows interaction with a trained sequence-to-sequence model through each stage of the translation process.
Semi-Amortized Variational Autoencoders
Amortized variational inference (AVI) replaces instance-specific local inference with a global inference network.
Ranked #2 on
Text Generation
on Yahoo Questions
Weightless: Lossy Weight Encoding For Deep Neural Network Compression
This results in up to a 1. 51x improvement over the state-of-the-art.
Dilated Convolutions for Modeling Long-Distance Genomic Dependencies
We consider the task of detecting regulatory elements in the human genome directly from raw DNA.
OpenNMT: Open-source Toolkit for Neural Machine Translation
We introduce an open-source toolkit for neural machine translation (NMT) to support research into model architectures, feature representations, and source modalities, while maintaining competitive performance, modularity and reasonable training requirements.
Adapting Sequence Models for Sentence Correction
In a controlled experiment of sequence-to-sequence approaches for the task of sentence correction, we find that character-based models are generally more effective than word-based models and models that encode subword information via convolutions, and that modeling the output data as a series of diffs improves effectiveness over standard approaches.
Challenges in Data-to-Document Generation
Recent neural models have shown significant progress on the problem of generating short descriptive texts conditioned on a small number of database records.
Adversarially Regularized Autoencoders
This adversarially regularized autoencoder (ARAE) allows us to generate natural textual outputs as well as perform manipulations in the latent space to induce change in the output space.
Structured Attention Networks
Attention networks have proven to be an effective approach for embedding categorical inference within a deep neural network.
OpenNMT: Open-Source Toolkit for Neural Machine Translation
We describe an open-source toolkit for neural machine translation (NMT).
Lie-Access Neural Turing Machines
The head is moved via Lie group actions, such as shifts or rotations, generated by a controller, and memory access is performed by linear smoothing in key space.
Image-to-Markup Generation with Coarse-to-Fine Attention
We present a neural encoder-decoder model to convert images into presentational markup based on a scalable coarse-to-fine attention mechanism.
Sequence-Level Knowledge Distillation
We demonstrate that standard knowledge distillation applied to word-level prediction can be effective for NMT, and also introduce two novel sequence-level versions of knowledge distillation that further improve performance, and somewhat surprisingly, seem to eliminate the need for beam search (even when applied on the original teacher model).
Ranked #1 on
Machine Translation
on IWSLT2015 Thai-English
LSTMVis: A Tool for Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks
In this work, we present LSTMVIS, a visual analysis tool for recurrent neural networks with a focus on understanding these hidden state dynamics.
Sequence-to-Sequence Learning as Beam-Search Optimization
In this work, we introduce a model and beam-search training scheme, based on the work of Daume III and Marcu (2005), that extends seq2seq to learn global sequence scores.
Ranked #13 on
Machine Translation
on IWSLT2015 German-English
Word Ordering Without Syntax
Recent work on word ordering has argued that syntactic structure is important, or even required, for effectively recovering the order of a sentence.
Sentence-Level Grammatical Error Identification as Sequence-to-Sequence Correction
We demonstrate that an attention-based encoder-decoder model can be used for sentence-level grammatical error identification for the Automated Evaluation of Scientific Writing (AESW) Shared Task 2016.
Learning Global Features for Coreference Resolution
There is compelling evidence that coreference prediction would benefit from modeling global information about entity-clusters.
Ranked #15 on
Coreference Resolution
on OntoNotes
A Neural Attention Model for Abstractive Sentence Summarization
Summarization based on text extraction is inherently limited, but generation-style abstractive methods have proven challenging to build.
Ranked #1 on
Extractive Text Summarization
on DUC 2004 Task 1
Character-Aware Neural Language Models
We describe a simple neural language model that relies only on character-level inputs.
Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks
One long-term goal of machine learning research is to produce methods that are applicable to reasoning and natural language, in particular building an intelligent dialogue agent.
A Tutorial on Dual Decomposition and Lagrangian Relaxation for Inference in Natural Language Processing
Dual decomposition, and more generally Lagrangian relaxation, is a classical method for combinatorial optimization; it has recently been applied to several inference problems in natural language processing (NLP).
