Search Results for author:
Found 44 papers, 22 papers with code
Gender Bias in Coreference Resolution
We present an empirical study of gender bias in coreference resolution systems.
The Universal Decompositional Semantics Dataset and Decomp Toolkit
We present the Universal Decompositional Semantics (UDS) dataset (v1. 0), which is bundled with the Decomp toolkit (v0. 1).
On Measuring Social Biases in Sentence Encoders
The Word Embedding Association Test shows that GloVe and word2vec word embeddings exhibit human-like implicit biases based on gender, race, and other social constructs (Caliskan et al., 2017).
Hypothesis Only Baselines in Natural Language Inference
We propose a hypothesis only baseline for diagnosing Natural Language Inference (NLI).
Thinking Like a Skeptic: Defeasible Inference in Natural Language
Defeasible inference is a mode of reasoning in which an inference (X is a bird, therefore X flies) may be weakened or overturned in light of new evidence (X is a penguin).
Neural-Davidsonian Semantic Proto-role Labeling
We present a model for semantic proto-role labeling (SPRL) using an adapted bidirectional LSTM encoding strategy that we call "Neural-Davidsonian": predicate-argument structure is represented as pairs of hidden states corresponding to predicate and argument head tokens of the input sequence.
Neural models of factuality
We present two neural models for event factuality prediction, which yield significant performance gains over previous models on three event factuality datasets: FactBank, UW, and MEANTIME.
Human Schema Curation via Causal Association Rule Mining
Event schemas are structured knowledge sources defining typical real-world scenarios (e. g., going to an airport).
"You are grounded!": Latent Name Artifacts in Pre-trained Language Models
Pre-trained language models (LMs) may perpetuate biases originating in their training corpus to downstream models.
SODAPOP: Open-Ended Discovery of Social Biases in Social Commonsense Reasoning Models
A common limitation of diagnostic tests for detecting social biases in NLP models is that they may only detect stereotypic associations that are pre-specified by the designer of the test.
Social Bias in Elicited Natural Language Inferences
We analyze the Stanford Natural Language Inference (SNLI) corpus in an investigation of bias and stereotyping in NLP data.
Theory-Grounded Measurement of U.S. Social Stereotypes in English Language Models
NLP models trained on text have been shown to reproduce human stereotypes, which can magnify harms to marginalized groups when systems are deployed at scale.
What do Large Language Models Learn about Scripts?
SIF is a two-staged framework that fine-tunes LM on a small set of ESD examples in the first stage.
It's Not Easy Being Wrong: Large Language Models Struggle with Process of Elimination Reasoning
Chain-of-thought (COT) prompting can help large language models (LLMs) reason toward correct answers, but its efficacy in reasoning toward incorrect answers is unexplored.
Artifacts or Abduction: How Do LLMs Answer Multiple-Choice Questions Without the Question?
We hope to motivate the use of stronger baselines in MCQA benchmarks, the design of robust MCQA datasets, and further efforts to explain LLM decision-making.
MedNLI Is Not Immune: Natural Language Inference Artifacts in the Clinical Domain
Crowdworker-constructed natural language inference (NLI) datasets have been found to contain statistical artifacts associated with the annotation process that allow hypothesis-only classifiers to achieve better-than-random performance (Poliak et al., 2018; Gururanganet et al., 2018; Tsuchiya, 2018).
Partial-input baselines show that NLI models can ignore context, but they don't
When strong partial-input baselines reveal artifacts in crowdsourced NLI datasets, the performance of full-input models trained on such datasets is often dismissed as reliance on spurious correlations.
Recognition of They/Them as Singular Personal Pronouns in Coreference Resolution
As using they/them as personal pronouns becomes increasingly common in English, it is important that coreference resolution systems work as well for individuals who use personal “they” as they do for those who use gendered personal pronouns.
Partial-input baselines show that NLI models can ignore context, but they don’t.
When strong partial-input baselines reveal artifacts in crowdsourced NLI datasets, the performance of full-input models trained on such datasets is often dismissed as reliance on spurious correlations.
Multilingual large language models leak human stereotypes across language boundaries
We evaluate human stereotypes and stereotypical associations manifested in multilingual large language models such as mBERT, mT5, and ChatGPT.
Collecting Diverse Natural Language Inference Problems for Sentence Representation Evaluation
We present a large-scale collection of diverse natural language inference (NLI) datasets that help provide insight into how well a sentence representation captures distinct types of reasoning.
Ordinal Common-sense Inference
Humans have the capacity to draw common-sense inferences from natural language: various things that are likely but not certain to hold based on established discourse, and are rarely stated explicitly.
Computational linking theory
A linking theory explains how verbs' semantic arguments are mapped to their syntactic arguments---the inverse of the Semantic Role Labeling task from the shallow semantic parsing literature.
Lexicosyntactic Inference in Neural Models
We use this dataset, which we make publicly available, to probe the behavior of current state-of-the-art neural systems, showing that these systems make certain systematic errors that are clearly visible through the lens of factuality prediction.
Cross-lingual Decompositional Semantic Parsing
We introduce the task of cross-lingual decompositional semantic parsing: mapping content provided in a source language into a decompositional semantic analysis based on a target language.
ParaBank: Monolingual Bitext Generation and Sentential Paraphrasing via Lexically-constrained Neural Machine Translation
We present ParaBank, a large-scale English paraphrase dataset that surpasses prior work in both quantity and quality.
Semantic Proto-Roles
We present the first large-scale, corpus based verification of Dowty{'}s seminal theory of proto-roles.
Causal Inference of Script Knowledge
When does a sequence of events define an everyday scenario and how can this knowledge be induced from text?
``You are grounded!'': Latent Name Artifacts in Pre-trained Language Models
Pre-trained language models (LMs) may perpetuate biases originating in their training corpus to downstream models.
Learning to Rationalize for Nonmonotonic Reasoning with Distant Supervision
In this paper, we investigate the extent to which neural models can reason about natural language rationales that explain model predictions, relying only on distant supervision with no additional annotation cost for human-written rationales.
Entailment Relation Aware Paraphrase Generation
We introduce a new task of entailment relation aware paraphrase generation which aims at generating a paraphrase conforming to a given entailment relation (e. g. equivalent, forward entailing, or reverse entailing) with respect to a given input.
Agent-Specific Deontic Modality Detection in Legal Language
Legal documents are typically long and written in legalese, which makes it particularly difficult for laypeople to understand their rights and duties.
What to Read in a Contract? Party-Specific Summarization of Legal Obligations, Entitlements, and Prohibitions
In this work, we propose a new task of party-specific extractive summarization for legal contracts to facilitate faster reviewing and improved comprehension of rights and duties.
Nichelle and Nancy: The Influence of Demographic Attributes and Tokenization Length on First Name Biases
We find that demographic attributes of a name (race, ethnicity, and gender) and name tokenization length are both factors that systematically affect the behavior of social commonsense reasoning models.
Pregnant Questions: The Importance of Pragmatic Awareness in Maternal Health Question Answering
Questions posed by information-seeking users often contain implicit false or potentially harmful assumptions.
How often are errors in natural language reasoning due to paraphrastic variability?
We propose a metric for evaluating the paraphrastic consistency of natural language reasoning models based on the probability of a model achieving the same correctness on two paraphrases of the same problem.
