Search Results for author:
Found 25 papers, 15 papers with code
Overview and Insights from the Shared Tasks at Scholarly Document Processing 2020: CL-SciSumm, LaySumm and LongSumm
We present the results of three Shared Tasks held at the Scholarly Document Processing Workshop at EMNLP2020: CL-SciSumm, LaySumm and LongSumm.
A Tale of Two Regulatory Regimes: Creation and Analysis of a Bilingual Privacy Policy Corpus
Over the past decade, researchers have started to explore the use of NLP to develop tools aimed at helping the public, vendors, and regulators analyze disclosures made in privacy policies.
Artifacts or Abduction: How Do LLMs Answer Multiple-Choice Questions Without the Question?
We hope to motivate the use of stronger baselines in MCQA benchmarks, the design of robust MCQA datasets, and further efforts to explain LLM decision-making.
OLMo: Accelerating the Science of Language Models
Given the importance of these details in scientifically studying these models, including their biases and potential risks, we believe it is essential for the research community to have access to powerful, truly open LMs.
Dolma: an Open Corpus of Three Trillion Tokens for Language Model Pretraining Research
Language models have become a critical technology to tackling a wide range of natural language processing tasks, yet many details about how the best-performing language models were developed are not reported.
The Unlocking Spell on Base LLMs: Rethinking Alignment via In-Context Learning
We analyze the effect of alignment tuning by examining the token distribution shift between base LLMs and their aligned counterpart.
MacGyver: Are Large Language Models Creative Problem Solvers?
We explore the creative problem-solving capabilities of modern LLMs in a novel constrained setting.
Agent Lumos: Unified and Modular Training for Open-Source Language Agents
To foster generalizable agent learning, we collect large-scale, unified, and high-quality training annotations derived from diverse ground-truth reasoning rationales across various complex interactive tasks.
What's In My Big Data?
We open-source WIMBD's code and artifacts to provide a standard set of evaluations for new text-based corpora and to encourage more analyses and transparency around them.
The Generative AI Paradox: "What It Can Create, It May Not Understand"
Specifically, we propose and test the Generative AI Paradox hypothesis: generative models, having been trained directly to reproduce expert-like outputs, acquire generative capabilities that are not contingent upon -- and can therefore exceed -- their ability to understand those same types of outputs.
Inference-Time Policy Adapters (IPA): Tailoring Extreme-Scale LMs without Fine-tuning
While extreme-scale language models have demonstrated exceptional performance on a variety of language tasks, the degree of control over these language models through pure prompting can often be limited.
CONDAQA: A Contrastive Reading Comprehension Dataset for Reasoning about Negation
We also have workers make three kinds of edits to the passage -- paraphrasing the negated statement, changing the scope of the negation, and reversing the negation -- resulting in clusters of question-answer pairs that are difficult for models to answer with spurious shortcuts.
Measuring Causal Effects of Data Statistics on Language Model's `Factual' Predictions
Our causal framework and our results demonstrate the importance of studying datasets and the benefits of causality for understanding NLP models.
Breaking Down Walls of Text: How Can NLP Benefit Consumer Privacy?
Privacy plays a crucial role in preserving democratic ideals and personal autonomy.
CURIE: An Iterative Querying Approach for Reasoning About Situations
Recently, models have been shown to predict the effects of unexpected situations, e. g., would cloudy skies help or hinder plant growth?
NoiseQA: Challenge Set Evaluation for User-Centric Question Answering
When Question-Answering (QA) systems are deployed in the real world, users query them through a variety of interfaces, such as speaking to voice assistants, typing questions into a search engine, or even translating questions to languages supported by the QA system.
Measuring and Improving Consistency in Pretrained Language Models
In this paper we study the question: Are Pretrained Language Models (PLMs) consistent with respect to factual knowledge?
On the Systematicity of Probing Contextualized Word Representations: The Case of Hypernymy in BERT
In particular, we demonstrate through a simple consistency probe that the ability to correctly retrieve hypernyms in cloze tasks, as used in prior work, does not correspond to systematic knowledge in BERT.
EIGEN: Event Influence GENeration using Pre-trained Language Models
Reasoning about events and tracking their influences is fundamental to understanding processes.
Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?
Although neural models have achieved impressive results on several NLP benchmarks, little is understood about the mechanisms they use to perform language tasks.
Question Answering for Privacy Policies: Combining Computational and Legal Perspectives
The PrivacyQA corpus offers a challenging corpus for question answering, with genuine real-world utility.
EQUATE: A Benchmark Evaluation Framework for Quantitative Reasoning in Natural Language Inference
Quantitative reasoning is a higher-order reasoning skill that any intelligent natural language understanding system can reasonably be expected to handle.
Stress Test Evaluation for Natural Language Inference
Natural language inference (NLI) is the task of determining if a natural language hypothesis can be inferred from a given premise in a justifiable manner.
Natural Language Inference
Natural Language Understanding
+1
Preserving Intermediate Objectives: One Simple Trick to Improve Learning for Hierarchical Models
In this work, we focus on improving learning for such hierarchical models and demonstrate our method on the task of speaker trait prediction.
Does the Geometry of Word Embeddings Help Document Classification? A Case Study on Persistent Homology Based Representations
We investigate the pertinence of methods from algebraic topology for text data analysis.
