Search Results for author:
Found 80 papers, 32 papers with code
Right for the Right Reason: Evidence Extraction for Trustworthy Tabular Reasoning
On the downstream tabular inference task, using only the automatically extracted evidence as the premise, our approach outperforms prior benchmarks.
Putting Words in BERT’s Mouth: Navigating Contextualized Vector Spaces with Pseudowords
We present a method for exploring regions around individual points in a contextualized vector space (particularly, BERT space), as a way to investigate how these regions correspond to word senses.
Automatic Entity State Annotation using the VerbNet Semantic Parser
Tracking entity states is a natural language processing task assumed to require human annotation.
Putting Context in SNACS: A 5-Way Classification of Adpositional Pragmatic Markers
The SNACS framework provides a network of semantic labels called supersenses for annotating adpositional semantics in corpora.
Psychotherapy is Not One Thing: Simultaneous Modeling of Different Therapeutic Approaches
Yet, NLP work in this area has been restricted to evaluating a single approach to treatment, when prior research indicates therapists used a wide variety of interventions with their clients, often in the same session.
Sprucing up Supersenses: Untangling the Semantic Clusters of Accompaniment and Purpose
We reevaluate an existing adpositional annotation scheme with respect to two thorny semantic domains: accompaniment and purpose.
Understanding the Logic of Direct Preference Alignment through Logic
Recent direct preference alignment algorithms (DPA), such as DPO, have shown great promise in aligning large language models to human preferences.
State Space Models are Strong Text Rerankers
Transformers dominate NLP and IR; but their inference inefficiencies and challenges in extrapolating to longer contexts have sparked interest in alternative model architectures.
Beyond Perplexity: Multi-dimensional Safety Evaluation of LLM Compression
To this end, we investigate the impact of model compression along four dimensions: (1) degeneration harm, i. e., bias and toxicity in generation; (2) representational harm, i. e., biases in discriminative tasks; (3) dialect bias; and(4) language modeling and downstream task performance.
An Empirical Investigation of Matrix Factorization Methods for Pre-trained Transformers
The increasing size of transformer-based models in NLP makes the question of compressing them important.
In-Context Example Ordering Guided by Label Distributions
Inspired by the idea of learning from label proportions, we propose two principles for in-context example ordering guided by model's probability predictions.
Promptly Predicting Structures: The Return of Inference
Can the promise of the prompt-based paradigm be extended to such structured outputs?
Whispers of Doubt Amidst Echoes of Triumph in NLP Robustness
Do larger and more performant models resolve NLP's longstanding robustness issues?
Measuring and Improving Attentiveness to Partial Inputs with Counterfactuals
Previous work has found that datasets with paired inputs are prone to correlations between a specific part of the input (e. g., the hypothesis in NLI) and the label; consequently, models trained only on those outperform chance.
TempTabQA: Temporal Question Answering for Semi-Structured Tables
Given these results, our dataset has the potential to serve as a challenging benchmark to improve the temporal reasoning capabilities of NLP models.
The Integer Linear Programming Inference Cookbook
At the end, we will see two worked examples to illustrate the use of these recipes.
Don't Retrain, Just Rewrite: Countering Adversarial Perturbations by Rewriting Text
For example, on sentiment classification using the SST-2 dataset, our method improves the adversarial accuracy over the best existing defense approach by more than 4% with a smaller decrease in task accuracy (0. 5% vs 2. 5%).
Learning Semantic Role Labeling from Compatible Label Sequences
In this paper, we eliminate such issue with a framework that jointly models VerbNet and PropBank labels as one sequence.
An In-depth Investigation of User Response Simulation for Conversational Search
Our goal is to supplement existing work with an insightful hand-analysis of unsolved challenges by the baseline and propose our solutions.
ClarifyDelphi: Reinforced Clarification Questions with Defeasibility Rewards for Social and Moral Situations
Context is everything, even in commonsense moral reasoning.
AGRO: Adversarial Discovery of Error-prone groups for Robust Optimization
We propose AGRO -- Adversarial Group discovery for Distributionally Robust Optimization -- an end-to-end approach that jointly identifies error-prone groups and improves accuracy on them.
Elaboration-Generating Commonsense Question Answering at Scale
In question answering requiring common sense, language models (e. g., GPT-3) have been used to generate text expressing background knowledge that helps improve performance.
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
BIG-bench focuses on tasks that are believed to be beyond the capabilities of current language models.
REFLACX, a dataset of reports and eye-tracking data for localization of abnormalities in chest x-rays
Furthermore, a small subset of the data contains readings from all radiologists, allowing for the calculation of inter-rater scores.
Putting Words in BERT's Mouth: Navigating Contextualized Vector Spaces with Pseudowords
We present a method for exploring regions around individual points in a contextualized vector space (particularly, BERT space), as a way to investigate how these regions correspond to word senses.
Is My Model Using The Right Evidence? Systematic Probes for Examining Evidence-Based Tabular Reasoning
Our experiments demonstrate that a RoBERTa-based model, representative of the current state-of-the-art, fails at reasoning on the following counts: it (a) ignores relevant parts of the evidence, (b) is over-sensitive to annotation artifacts, and (c) relies on the knowledge encoded in the pre-trained language model rather than the evidence presented in its tabular inputs.
Evaluating Relaxations of Logic for Neural Networks: A Comprehensive Study
Symbolic knowledge can provide crucial inductive bias for training neural models, especially in low data regimes.
A Closer Look at How Fine-tuning Changes BERT
Given the prevalence of pre-trained contextualized representations in today's NLP, there have been many efforts to understand what information they contain, and why they seem to be universally successful.
X-FACT: A New Benchmark Dataset for Multilingual Fact Checking
In this work, we introduce X-FACT: the largest publicly available multilingual dataset for factual verification of naturally existing real-world claims.
Database Workload Characterization with Query Plan Encoders
To address this workload characterization problem, we propose our query plan encoders that learn essential features and their correlations from query plans.
DirectProbe: Studying Representations without Classifiers
Understanding how linguistic structures are encoded in contextualized embedding could help explain their impressive performance across NLP@.
Incorporating External Knowledge to Enhance Tabular Reasoning
Reasoning about tabular information presents unique challenges to modern NLP approaches which largely rely on pre-trained contextualized embeddings of text.
VERB: Visualizing and Interpreting Bias Mitigation Techniques for Word Representations
To aid this, we present Visualization of Embedding Representations for deBiasing system ("VERB"), an open-source web-based visualization tool that helps the users gain a technical understanding and visual intuition of the inner workings of debiasing techniques, with a focus on their geometric properties.
BERT & Family Eat Word Salad: Experiments with Text Understanding
In this paper, we study the response of large models from the BERT family to incoherent inputs that should confuse any model that claims to understand natural language.
Supertagging the Long Tail with Tree-Structured Decoding of Complex Categories
Our best tagger is capable of recovering a sizeable fraction of the long-tail supertags and even generates CCG categories that have never been seen in training, while approximating the prior state of the art in overall tag accuracy with fewer parameters.
UnQovering Stereotyping Biases via Underspecified Questions
Our broad study reveals that (1) all these models, with and without fine-tuning, have notable stereotyping biases in these classes; (2) larger models often have higher bias; and (3) the effect of fine-tuning on bias varies strongly with the dataset and the model size.
A Simple Global Neural Discourse Parser
Discourse parsing is largely dominated by greedy parsers with manually-designed features, while global parsing is rare due to its computational expense.
OSCaR: Orthogonal Subspace Correction and Rectification of Biases in Word Embeddings
Language representations are known to carry stereotypical biases and, as a result, lead to biased predictions in downstream tasks.
Learning Constraints for Structured Prediction Using Rectifier Networks
Various natural language processing tasks are structured prediction problems where outputs are constructed with multiple interdependent decisions.
INFOTABS: Inference on Tables as Semi-structured Data
In this paper, we observe that semi-structured tabulated text is ubiquitous; understanding them requires not only comprehending the meaning of text fragments, but also implicit relationships between them.
Structured Tuning for Semantic Role Labeling
We start with a strong baseline (RoBERTa) to validate the impact of our approach, and show that our framework outperforms the baseline by learning to comply with declarative constraints.
Amazon at MRP 2019: Parsing Meaning Representations with Lexical and Phrasal Anchoring
This paper describes the system submission of our team Amazon to the shared task on Cross Framework Meaning Representation Parsing (MRP) at the 2019 Conference for Computational Language Learning (CoNLL).
On the Limits of Learning to Actively Learn Semantic Representations
We conclude that the current applicability of LTAL for improving data efficiency in learning semantic meaning representations is limited.
Learning Semantic Representations
Natural Language Understanding
A Logic-Driven Framework for Consistency of Neural Models
While neural models show remarkable accuracy on individual predictions, their internal beliefs can be inconsistent across examples.
On Measuring and Mitigating Biased Inferences of Word Embeddings
Word embeddings carry stereotypical connotations from the text they are trained on, which can lead to invalid inferences in downstream models that rely on them.
Preparing SNACS for Subjects and Objects
Research on adpositions and possessives in multiple languages has led to a small inventory of general-purpose meaning classes that disambiguate tokens.
Observing Dialogue in Therapy: Categorizing and Forecasting Behavioral Codes
Specifically, we address the problem of providing real-time guidance to therapists with a dialogue observer that (1) categorizes therapist and client MI behavioral codes and, (2) forecasts codes for upcoming utterances to help guide the conversation and potentially alert the therapist.
Augmenting Neural Networks with First-order Logic
Today, the dominant paradigm for training neural networks involves minimizing task loss on a large dataset.
Beyond Context: A New Perspective for Word Embeddings
We define a new modeling framework for training word embeddings that captures this intuition.
Learning In Practice: Reasoning About Quantization
There is a mismatch between the standard theoretical analyses of statistical machine learning and how learning is used in practice.
Visual Interrogation of Attention-Based Models for Natural Language Inference and Machine Comprehension
Neural networks models have gained unprecedented popularity in natural language processing due to their state-of-the-art performance and the flexible end-to-end training scheme.
Learning to Speed Up Structured Output Prediction
Predicting structured outputs can be computationally onerous due to the combinatorially large output spaces.
Comprehensive Supersense Disambiguation of English Prepositions and Possessives
Semantic relations are often signaled with prepositional or possessive marking--but extreme polysemy bedevils their analysis and automatic interpretation.
Ranked #4 on
Natural Language Understanding
on STREUSLE
(Role F1 (Preps) metric)
Newton: Gravitating Towards the Physical Limits of Crossbar Acceleration
Many recent works have designed accelerators for Convolutional Neural Networks (CNNs).
Double Trouble: The Problem of Construal in Semantic Annotation of Adpositions
We consider the semantics of prepositions, revisiting a broad-coverage annotation scheme used for annotating all 4, 250 preposition tokens in a 55, 000 word corpus of English.
An Algebra for Feature Extraction
Though feature extraction is a necessary first step in statistical NLP, it is often seen as a mere preprocessing step.
Adposition and Case Supersenses v2.6: Guidelines for English
This document offers a detailed linguistic description of SNACS (Semantic Network of Adposition and Case Supersenses; Schneider et al., 2018), an inventory of 52 semantic labels ("supersenses") that characterize the use of adpositions and case markers at a somewhat coarse level of granularity, as demonstrated in the STREUSLE corpus (https://github. com/nert-nlp/streusle/ ; version 4. 5 tracks guidelines version 2. 6).
Integer Linear Programming formulations in Natural Language Processing
We will cover a range of topics, from the theoretical foundations of learning and inference with ILP models, to practical modeling guides, to software packages and applications. The goal of this tutorial is to introduce the computational framework to broader ACL community, motivate it as a generic framework for learning and inference in global NLP decision problems, present some of the key theoretical and practical issues involved and survey some of the existing applications of it as a way to promote further development of the framework and additional applications.
Coping with Construals in Broad-Coverage Semantic Annotation of Adpositions
We consider the semantics of prepositions, revisiting a broad-coverage annotation scheme used for annotating all 4, 250 preposition tokens in a 55, 000 word corpus of English.
A corpus of preposition supersenses in English web reviews
We present the first corpus annotated with preposition supersenses, unlexicalized categories for semantic functions that can be marked by English prepositions (Schneider et al., 2015).
EDISON: Feature Extraction for NLP, Simplified
We present EDISON, a Java library of feature generation functions used in a suite of state-of-the-art NLP tools, based on a set of generic NLP data structures.
Expressiveness of Rectifier Networks
However, unlike threshold and sigmoid networks, ReLU networks are less explored from the perspective of their expressiveness.
IllinoisSL: A JAVA Library for Structured Prediction
IllinoisSL is a Java library for learning structured prediction models.
Learning Distributed Representations for Structured Output Prediction
In recent years, distributed representations of inputs have led to performance gains in many applications by allowing statistical information to be shared across inputs.
An Inventory of Preposition Relations
We describe an inventory of semantic relations that are expressed by prepositions.
Modeling Semantic Relations Expressed by Prepositions
This paper introduces the problem of predicting semantic relations expressed by prepositions and develops statistical learning models for predicting the relations, their arguments and the semantic types of the arguments.
An NLP Curator (or: How I Learned to Stop Worrying and Love NLP Pipelines)
Natural Language Processing continues to grow in popularity in a range of research and commercial applications, yet managing the wide array of potential NLP components remains a difficult problem.
