Search Results for author:
Found 54 papers, 25 papers with code
Investigating the Effect of Background Knowledge on Natural Questions
We create a subset of the NQ data, Factual Questions (FQ), where the questions have evidence in the KB in the form of paths that link question entities to answer entities but still must be answered using text, to facilitate further research into KB integration methods.
ReCaLL: Membership Inference via Relative Conditional Log-Likelihoods
We propose ReCaLL (Relative Conditional Log-Likelihood), a novel membership inference attack (MIA) to detect LLMs' pretraining data by leveraging their conditional language modeling capabilities.
Raccoon: Prompt Extraction Benchmark of LLM-Integrated Applications
With the proliferation of LLM-integrated applications such as GPT-s, millions are deployed, offering valuable services through proprietary instruction prompts.
Stratified Prediction-Powered Inference for Hybrid Language Model Evaluation
Prediction-powered inference (PPI) is a method that improves statistical estimates based on limited human-labeled data.
Atomic Self-Consistency for Better Long Form Generations
In this paper, we introduce Atomic Self-Consistency (ASC), a technique for improving the recall of relevant information in an LLM response.
Tailoring Vaccine Messaging with Common-Ground Opinions
We define the task of tailoring vaccine interventions to a Common-Ground Opinion (CGO).
Bayesian Prediction-Powered Inference
Prediction-powered inference (PPI) is a method that improves statistical estimates based on limited human-labeled data.
ChatShop: Interactive Information Seeking with Language Agents
The desire and ability to seek new information strategically are fundamental to human learning but often overlooked in current language agent evaluation.
IsoBench: Benchmarking Multimodal Foundation Models on Isomorphic Representations
Current foundation models exhibit impressive capabilities when prompted either with text only or with both image and text inputs.
Extracting Polymer Nanocomposite Samples from Full-Length Documents
This paper investigates the use of large language models (LLMs) for extracting sample lists of polymer nanocomposites (PNCs) from full-length materials science research papers.
Adversarial Math Word Problem Generation
Large language models (LLMs) have significantly transformed the educational landscape.
Calibrating Long-form Generations from Large Language Models
Addressing this gap, we introduce a unified calibration framework, in which both the correctness of the LLMs' responses and their associated confidence levels are treated as distributions across a range of scores.
Hierarchical Multi-Label Classification of Online Vaccine Concerns
Vaccine concerns are an ever-evolving target, and can shift quickly as seen during the COVID-19 pandemic.
Hierarchical Multi-Instance Multi-Label Learning for Detecting Propaganda Techniques
Since the introduction of the SemEval 2020 Task 11 (Martino et al., 2020a), several approaches have been proposed in the literature for classifying propaganda based on the rhetorical techniques used to influence readers.
Selectively Answering Ambiguous Questions
We investigate question answering from this perspective, focusing on answering a subset of questions with a high degree of accuracy, from a set of questions in which many are inherently ambiguous.
Salient Span Masking for Temporal Understanding
First, we find that SSM alone improves the downstream performance on three temporal tasks by an avg.
Learning the Legibility of Visual Text Perturbations
In this work, we address this gap by learning models that predict the legibility of a perturbed string, and rank candidate perturbations based on their legibility.
DIFFQG: Generating Questions to Summarize Factual Changes
We propose representing factual changes between paired documents as question-answer pairs, where the answer to the same question differs between two versions.
On the State of the Art in Authorship Attribution and Authorship Verification
Despite decades of research on authorship attribution (AA) and authorship verification (AV), inconsistent dataset splits/filtering and mismatched evaluation methods make it difficult to assess the state of the art.
Characterizing the Efficiency vs. Accuracy Trade-off for Long-Context NLP Models
In this work, we perform a systematic study of this accuracy vs. efficiency trade-off on two widely used long-sequence models - Longformer-Encoder-Decoder (LED) and Big Bird - during fine-tuning and inference on four datasets from the SCROLLS benchmark.
ASQA: Factoid Questions Meet Long-Form Answers
In contrast to existing long-form QA tasks (such as ELI5), ASQA admits a clear notion of correctness: a user faced with a good summary should be able to answer different interpretations of the original ambiguous question.
Time-Aware Language Models as Temporal Knowledge Bases
We introduce a diagnostic dataset aimed at probing LMs for factual knowledge that changes over time and highlight problems with LMs at either end of the spectrum -- those trained on specific slices of temporal data, as well as those trained on a wide range of temporal data.
Fool Me Twice: Entailment from Wikipedia Gamification
We release FoolMeTwice (FM2 for short), a large dataset of challenging entailment pairs collected through a fun multi-player game.
Reasoning Over Virtual Knowledge Bases With Open Predicate Relations
We present the Open Predicate Query Language (OPQL); a method for constructing a virtual KB (VKB) trained entirely from text.
Evaluating Explanations: How much do explanations from the teacher aid students?
While many methods purport to explain predictions by highlighting salient features, what aims these explanations serve and how they ought to be evaluated often go unstated.
Weakly- and Semi-supervised Evidence Extraction
For many prediction tasks, stakeholders desire not only predictions but also supporting evidence that a human can use to verify its correctness.
Differentiable Open-Ended Commonsense Reasoning
As a step towards making commonsense reasoning research more realistic, we propose to study open-ended commonsense reasoning (OpenCSR) -- the task of answering a commonsense question without any pre-defined choices -- using as a resource only a corpus of commonsense facts written in natural language.
ToTTo: A Controlled Table-To-Text Generation Dataset
We present ToTTo, an open-domain English table-to-text dataset with over 120, 000 training examples that proposes a controlled generation task: given a Wikipedia table and a set of highlighted table cells, produce a one-sentence description.
Ranked #3 on
Data-to-Text Generation
on ToTTo
Differentiable Reasoning over a Virtual Knowledge Base
In particular, we describe a neural module, DrKIT, that traverses textual data like a KB, softly following paths of relations between mentions of entities in the corpus.
Learning to Deceive with Attention-Based Explanations
Attention mechanisms are ubiquitous components in neural architectures applied to natural language processing.
PubMedQA: A Dataset for Biomedical Research Question Answering
We introduce PubMedQA, a novel biomedical question answering (QA) dataset collected from PubMed abstracts.
Ranked #8 on
Question Answering
on PubMedQA
Handling Divergent Reference Texts when Evaluating Table-to-Text Generation
Automatically constructed datasets for generating text from semi-structured data (tables), such as WikiBio, often contain reference texts that diverge from the information in the corresponding semi-structured data.
Combating Adversarial Misspellings with Robust Word Recognition
To combat adversarial spelling mistakes, we propose placing a word recognition model in front of the downstream classifier.
Text Generation with Exemplar-based Adaptive Decoding
We propose a novel conditioned text generation model.
Probing Biomedical Embeddings from Language Models
For this we use the pre-trained LMs as fixed feature extractors and restrict the downstream task models to not have additional sequence modeling layers.
GLoMo: Unsupervised Learning of Transferable Relational Graphs
We also show that the learned graphs are generic enough to be transferred to different embeddings on which the graphs have not been trained (including GloVe embeddings, ELMo embeddings, and task-specific RNN hidden units), or embedding-free units such as image pixels.
AttentionMeSH: Simple, Effective and Interpretable Automatic MeSH Indexer
There are millions of articles in PubMed database.
Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text
In this paper we look at a more practical setting, namely QA over the combination of a KB and entity-linked text, which is appropriate when an incomplete KB is available with a large text corpus.
Graph Representation Learning
Open-Domain Question Answering
GLoMo: Unsupervisedly Learned Relational Graphs as Transferable Representations
We also show that the learned graphs are generic enough to be transferred to different embeddings on which the graphs have not been trained (including GloVe embeddings, ELMo embeddings, and task-specific RNN hidden unit), or embedding-free units such as image pixels.
Embedding Text in Hyperbolic Spaces
Ideally, we could incorporate our prior knowledge of this hierarchical structure into unsupervised learning algorithms that work on text data.
Neural Models for Reasoning over Multiple Mentions using Coreference
Many problems in NLP require aggregating information from multiple mentions of the same entity which may be far apart in the text.
Ranked #7 on
Question Answering
on WikiHop
Simple and Effective Semi-Supervised Question Answering
Recent success of deep learning models for the task of extractive Question Answering (QA) is hinged on the availability of large annotated corpora.
Quasar: Datasets for Question Answering by Search and Reading
ClueWeb09 serves as the background corpus for extracting these answers.
Question Answering from Unstructured Text by Retrieval and Comprehension
Open domain Question Answering (QA) systems must interact with external knowledge sources, such as web pages, to find relevant information.
Linguistic Knowledge as Memory for Recurrent Neural Networks
We introduce a model that encodes such graphs as explicit memory in recurrent neural networks, and use it to model coreference relations in text.
Ranked #1 on
Question Answering
on CNN / Daily Mail
Using Graphs of Classifiers to Impose Declarative Constraints on Semi-supervised Learning
We propose a general approach to modeling semi-supervised learning (SSL) algorithms.
A Comparative Study of Word Embeddings for Reading Comprehension
The focus of past machine learning research for Reading Comprehension tasks has been primarily on the design of novel deep learning architectures.
A User Simulator for Task-Completion Dialogues
Then, one can train reinforcement learning agents in an online fashion as they interact with the simulator.
Words or Characters? Fine-grained Gating for Reading Comprehension
Previous work combines word-level and character-level representations using concatenation or scalar weighting, which is suboptimal for high-level tasks like reading comprehension.
Ranked #50 on
Question Answering
on SQuAD1.1 dev
Towards End-to-End Reinforcement Learning of Dialogue Agents for Information Access
In this paper, we address this limitation by replacing symbolic queries with an induced "soft" posterior distribution over the KB that indicates which entities the user is interested in.
Bootstrapping Distantly Supervised IE using Joint Learning and Small Well-structured Corpora
We propose a framework to improve performance of distantly-supervised relation extraction, by jointly learning to solve two related tasks: concept-instance extraction and relation extraction.
Gated-Attention Readers for Text Comprehension
In this paper we study the problem of answering cloze-style questions over documents.
Ranked #1 on
Question Answering
on Children's Book Test
Tweet2Vec: Character-Based Distributed Representations for Social Media
Text from social media provides a set of challenges that can cause traditional NLP approaches to fail.
