Search Results for author:
Found 58 papers, 26 papers with code
Automatic Selection of Context Configurations for Improved Class-Specific Word Representations
With our selected context configurations, we train on only 14% (A), 26. 2% (V), and 33. 6% (N) of all dependency-based contexts, resulting in a reduced training time.
The Effect of Different Writing Tasks on Linguistic Style: A Case Study of the ROC Story Cloze Task
A writer's style depends not just on personal traits but also on her intent and mental state.
Story Cloze Task: UW NLP System
This paper describes University of Washington NLP{'}s submission for the Linking Models of Lexical, Sentential and Discourse-level Semantics (LSDSem 2017) shared task{---}the Story Cloze Task.
Annotation Artifacts in Natural Language Inference Data
Large-scale datasets for natural language inference are created by presenting crowd workers with a sentence (premise), and asking them to generate three new sentences (hypotheses) that it entails, contradicts, or is logically neutral with respect to.
A Dataset of Peer Reviews (PeerRead): Collection, Insights and NLP Applications
In the first task, we show that simple models can predict whether a paper is accepted with up to 21% error reduction compared to the majority baseline.
SoPa: Bridging CNNs, RNNs, and Weighted Finite-State Machines
Recurrent and convolutional neural networks comprise two distinct families of models that have proven to be useful for encoding natural language utterances.
Explainable artificial intelligence
General Classification
+3
LSTMs Exploit Linguistic Attributes of Data
While recurrent neural networks have found success in a variety of natural language processing applications, they are general models of sequential data.
Bridging CNNs, RNNs, and Weighted Finite-State Machines
Recurrent and convolutional neural networks comprise two distinct families of models that have proven to be useful for encoding natural language utterances.
SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference
Given a partial description like "she opened the hood of the car," humans can reason about the situation and anticipate what might come next ("then, she examined the engine").
Ranked #4 on
Common Sense Reasoning
on SWAG
Rational Recurrences
We characterize this connection formally, defining rational recurrences to be recurrent hidden state update functions that can be written as the Forward calculation of a finite set of WFSAs.
Inoculation by Fine-Tuning: A Method for Analyzing Challenge Datasets
Several datasets have recently been constructed to expose brittleness in models trained on existing benchmarks.
PaLM: A Hybrid Parser and Language Model
We present PaLM, a hybrid parser and neural language model.
Show Your Work: Improved Reporting of Experimental Results
Research in natural language processing proceeds, in part, by demonstrating that new models achieve superior performance (e. g., accuracy) on held-out test data, compared to previous results.
RNN Architecture Learning with Sparse Regularization
Our method also highlights the interpretable properties of rational RNNs.
Knowledge Enhanced Contextual Word Representations
Contextual word representations, typically trained on unstructured, unlabeled text, do not contain any explicit grounding to real world entities and are often unable to remember facts about those entities.
Ranked #9 on
Relation Classification
on TACRED
Fair Correlation Clustering
We consider two variations of fairness constraint for the problem of correlation clustering where each node has a color, and the goal is to form clusters that do not over-represent vertices of any color.
Fine-Tuning Pretrained Language Models: Weight Initializations, Data Orders, and Early Stopping
We publicly release all of our experimental data, including training and validation scores for 2, 100 trials, to encourage further analysis of training dynamics during fine-tuning.
The Right Tool for the Job: Matching Model and Instance Complexities
Our method presents a favorable speed/accuracy tradeoff in almost all cases, producing models which are up to five times faster than the state of the art, while preserving their accuracy.
A Formal Hierarchy of RNN Architectures
While formally extending these findings to unsaturated RNNs is left to future work, we hypothesize that the practical learnable capacity of unsaturated RNNs obeys a similar hierarchy.
A Mixture of $h-1$ Heads is Better than $h$ Heads
Multi-head attentive neural architectures have achieved state-of-the-art results on a variety of natural language processing tasks.
A Mixture of h - 1 Heads is Better than h Heads
Multi-head attentive neural architectures have achieved state-of-the-art results on a variety of natural language processing tasks.
Dataset Cartography: Mapping and Diagnosing Datasets with Training Dynamics
Experiments across four datasets show that these model-dependent measures reveal three distinct regions in the data map, each with pronounced characteristics.
Extracting a Knowledge Base of Mechanisms from COVID-19 Papers
The COVID-19 pandemic has spawned a diverse body of scientific literature that is challenging to navigate, stimulating interest in automated tools to help find useful knowledge.
Effects of Parameter Norm Growth During Transformer Training: Inductive Bias from Gradient Descent
To better understand this bias, we study the tendency for transformer parameters to grow in magnitude ($\ell_2$ norm) during training, and its implications for the emergent representations within self attention layers.
A Refined Analysis of Submodular Greedy
Many algorithms for maximizing a monotone submodular function subject to a knapsack constraint rely on the natural greedy heuristic.
Data Structures and Algorithms
Random Feature Attention
RFA can be used as a drop-in replacement for conventional softmax attention and offers a straightforward way of learning with recency bias through an optional gating mechanism.
Ranked #27 on
Machine Translation
on IWSLT2014 German-English
Automatic Generation of Contrast Sets from Scene Graphs: Probing the Compositional Consistency of GQA
Recent works have shown that supervised models often exploit data artifacts to achieve good test scores while their performance severely degrades on samples outside their training distribution.
Provable Limitations of Acquiring Meaning from Ungrounded Form: What Will Future Language Models Understand?
We study whether assertions enable a system to emulate representations preserving semantic relations like equivalence.
Data Efficient Masked Language Modeling for Vision and Language
We investigate a range of alternative masking strategies specific to the cross-modal setting that address these shortcomings, aiming for better fusion of text and image in the learned representation.
Expected Validation Performance and Estimation of a Random Variable's Maximum
We find that the two biased estimators lead to the fewest incorrect conclusions, which hints at the importance of minimizing variance and MSE.
ABC: Attention with Bounded-memory Control
One way to improve the efficiency is to bound the memory size.
Data Contamination: From Memorization to Exploitation
Experiments with two models and three downstream tasks show that exploitation exists in some cases, but in others the models memorize the contaminated data, but do not exploit it.
A deep learning framework for the detection and quantification of drusen and reticular pseudodrusen on optical coherence tomography
Methods - A DL framework was developed consisting of a classification model and an out-of-distribution (OOD) detection model for the identification of ungradable scans; a classification model to identify scans with drusen or RPD; and an image segmentation model to independently segment lesions as RPD or drusen.
TangoBERT: Reducing Inference Cost by using Cascaded Architecture
In order to reduce this computational load in inference time, we present TangoBERT, a cascaded model architecture in which instances are first processed by an efficient but less accurate first tier model, and only part of those instances are additionally processed by a less efficient but more accurate second tier model.
On the Limitations of Dataset Balancing: The Lost Battle Against Spurious Correlations
Recent work has shown that deep learning models in NLP are highly sensitive to low-level correlations between simple features and specific output labels, leading to overfitting and lack of generalization.
Measuring the Carbon Intensity of AI in Cloud Instances
By providing unprecedented access to computational resources, cloud computing has enabled rapid growth in technologies such as machine learning, the computational demands of which incur a high energy cost and a commensurate carbon footprint.
Fewer Errors, but More Stereotypes? The Effect of Model Size on Gender Bias
We find that while larger models outperform smaller ones, the probability that their mistakes are caused by gender bias is higher.
WinoGAViL: Gamified Association Benchmark to Challenge Vision-and-Language Models
While vision-and-language models perform well on tasks such as visual question answering, they struggle when it comes to basic human commonsense reasoning skills.
Ranked #1 on
Common Sense Reasoning
on WinoGAViL
Efficient Methods for Natural Language Processing: A Survey
Recent work in natural language processing (NLP) has yielded appealing results from scaling model parameters and training data; however, using only scale to improve performance means that resource consumption also grows.
How Much Does Attention Actually Attend? Questioning the Importance of Attention in Pretrained Transformers
Our results motivate research on simpler alternatives to input-dependent attention, as well as on methods for better utilization of this mechanism in the Transformer architecture.
VASR: Visual Analogies of Situation Recognition
We leverage situation recognition annotations and the CLIP model to generate a large set of 500k candidate analogies.
Ranked #1 on
Visual Reasoning
on VASR
Breaking Common Sense: WHOOPS! A Vision-and-Language Benchmark of Synthetic and Compositional Images
We introduce WHOOPS!, a new dataset and benchmark for visual commonsense.
Ranked #1 on
Image-to-Text Retrieval
on WHOOPS! A Vision-and-Language Benchmark of Synthetic and Compositional Images
(using extra training data)
Textually Pretrained Speech Language Models
In this work, we propose TWIST, a method for training SpeechLMs using a warm-start from a pretrained textual language models.
Fighting Bias with Bias: Promoting Model Robustness by Amplifying Dataset Biases
We suggest that in order to drive the development of models robust to subtle biases, dataset biases should be amplified in the training set.
Finding the SWEET Spot: Analysis and Improvement of Adaptive Inference in Low Resource Settings
Adaptive inference is a simple method for reducing inference costs.
Morphosyntactic probing of multilingual BERT models
We introduce an extensive dataset for multilingual probing of morphological information in language models (247 tasks across 42 languages from 10 families), each consisting of a sentence with a target word and a morphological tag as the desired label, derived from the Universal Dependencies treebanks.
Surveying (Dis)Parities and Concerns of Compute Hungry NLP Research
Many recent improvements in NLP stem from the development and use of large pre-trained language models (PLMs) with billions of parameters.
Read, Look or Listen? What's Needed for Solving a Multimodal Dataset
We propose a two-step method to analyze multimodal datasets, which leverages a small seed of human annotation to map each multimodal instance to the modalities required to process it.
Transformers are Multi-State RNNs
We further show that pretrained transformers can be converted into $\textit{finite}$ multi-state RNNs by fixing the size of their hidden state.
The Larger the Better? Improved LLM Code-Generation via Budget Reallocation
On the other hand, in scenarios where unit-tests are unavailable, a ranking-based selection of candidates from the smaller model falls short of the performance of a single output from larger ones.
Expected Validation Performance and Estimation of a Random Variable’s Maximum
We find that the two biased estimators lead to the fewest incorrect conclusions, which hints at the importance of minimizing variance and MSE.
