Search Results for author:
Found 85 papers, 31 papers with code
Efficient Relational Learning with Hidden Variable Detection
Markov networks (MNs) can incorporate arbitrarily complex features in modeling relational data.
Programming with Personalized PageRank: A Locally Groundable First-Order Probabilistic Logic
In many probabilistic first-order representation systems, inference is performed by "grounding"---i. e., mapping it to a propositional representation, and then performing propositional inference.
The Effect of Biased Communications On Both Trusting and Suspicious Voters
We consider scenarios in which this effect arises in a model of rational decision making which includes the possibility of deceptive information.
Exploratory Learning
In multiclass semi-supervised learning (SSL), it is sometimes the case that the number of classes present in the data is not known, and hence no labeled examples are provided for some classes.
WebSets: Extracting Sets of Entities from the Web Using Unsupervised Information Extraction
We describe a open-domain information extraction method for extracting concept-instance pairs from an HTML corpus.
Efficient Inference and Learning in a Large Knowledge Base: Reasoning with Extracted Information using a Locally Groundable First-Order Probabilistic Logic
We show that the problem of constructing proofs for this logic is related to computation of personalized PageRank (PPR) on a linearized version of the proof space, and using on this connection, we develop a proveably-correct approximate grounding scheme, based on the PageRank-Nibble algorithm.
Grounded Discovery of Coordinate Term Relationships between Software Entities
To this end, we develop a similarity measure for Java classes using distributional information about how they are used in software, which we combine with corpus statistics on the distribution of contexts in which the classes appear in text.
Distant IE by Bootstrapping Using Lists and Document Structure
Distant labeling for information extraction (IE) suffers from noisy training data.
Semantic Scan: Detecting Subtle, Spatially Localized Events in Text Streams
Many methods have been proposed for detecting emerging events in text streams using topic modeling.
Revisiting Semi-Supervised Learning with Graph Embeddings
We present a semi-supervised learning framework based on graph embeddings.
Ranked #2 on
Node Classification
on USA Air-Traffic
Tweet2Vec: Character-Based Distributed Representations for Social Media
Text from social media provides a set of challenges that can cause traditional NLP approaches to fail.
TensorLog: A Differentiable Deductive Database
Then, for each type of query to the factor graph, the message-passing steps required to perform belief propagation (BP) are "unrolled" into a function, which is differentiable.
Review Networks for Caption Generation
We propose a novel extension of the encoder-decoder framework, called a review network.
Gated-Attention Readers for Text Comprehension
In this paper we study the problem of answering cloze-style questions over documents.
Ranked #1 on
Question Answering
on Children's Book Test
Bootstrapping Distantly Supervised IE using Joint Learning and Small Well-structured Corpora
We propose a framework to improve performance of distantly-supervised relation extraction, by jointly learning to solve two related tasks: concept-instance extraction and relation extraction.
Words or Characters? Fine-grained Gating for Reading Comprehension
Previous work combines word-level and character-level representations using concatenation or scalar weighting, which is suboptimal for high-level tasks like reading comprehension.
Ranked #50 on
Question Answering
on SQuAD1.1 dev
Semi-Supervised QA with Generative Domain-Adaptive Nets
In this framework, we train a generative model to generate questions based on the unlabeled text, and combine model-generated questions with human-generated questions for training question answering models.
Differentiable Learning of Logical Rules for Knowledge Base Reasoning
We propose a framework, Neural Logic Programming, that combines the parameter and structure learning of first-order logical rules in an end-to-end differentiable model.
A Comparative Study of Word Embeddings for Reading Comprehension
The focus of past machine learning research for Reading Comprehension tasks has been primarily on the design of novel deep learning architectures.
Using Graphs of Classifiers to Impose Declarative Constraints on Semi-supervised Learning
We propose a general approach to modeling semi-supervised learning (SSL) algorithms.
Linguistic Knowledge as Memory for Recurrent Neural Networks
We introduce a model that encodes such graphs as explicit memory in recurrent neural networks, and use it to model coreference relations in text.
Ranked #1 on
Question Answering
on CNN / Daily Mail
Transfer Learning for Sequence Tagging with Hierarchical Recurrent Networks
Recent papers have shown that neural networks obtain state-of-the-art performance on several different sequence tagging tasks.
Ranked #10 on
Part-Of-Speech Tagging
on Penn Treebank
Good Semi-supervised Learning that Requires a Bad GAN
Semi-supervised learning methods based on generative adversarial networks (GANs) obtained strong empirical results, but it is not clear 1) how the discriminator benefits from joint training with a generator, and 2) why good semi-supervised classification performance and a good generator cannot be obtained at the same time.
Quasar: Datasets for Question Answering by Search and Reading
ClueWeb09 serves as the background corpus for extracting these answers.
TensorLog: Deep Learning Meets Probabilistic DBs
We present an implementation of a probabilistic first-order logic called TensorLog, in which classes of logical queries are compiled into differentiable functions in a neural-network infrastructure such as Tensorflow or Theano.
Breaking the Softmax Bottleneck: A High-Rank RNN Language Model
We formulate language modeling as a matrix factorization problem, and show that the expressiveness of Softmax-based models (including the majority of neural language models) is limited by a Softmax bottleneck.
Ranked #11 on
Language Modelling
on Penn Treebank (Word Level)
Learning to Organize Knowledge and Answer Questions with N-Gram Machines
Though deep neural networks have great success in natural language processing, they are limited at more knowledge intensive AI tasks, such as open-domain Question Answering (QA).
LEARNING TO ORGANIZE KNOWLEDGE WITH N-GRAM MACHINES
Existing end-to-end deep QA models (Miller et al., 2016; Weston et al., 2014) need to read the entire text after observing the question, and therefore their complexity in responding a question is linear in the text size.
Neural Models for Reasoning over Multiple Mentions using Coreference
Many problems in NLP require aggregating information from multiple mentions of the same entity which may be far apart in the text.
Ranked #7 on
Question Answering
on WikiHop
Semi-Supervised Learning with Declaratively Specified Entropy Constraints
We propose a technique for declaratively specifying strategies for semi-supervised learning (SSL).
GLoMo: Unsupervisedly Learned Relational Graphs as Transferable Representations
We also show that the learned graphs are generic enough to be transferred to different embeddings on which the graphs have not been trained (including GloVe embeddings, ELMo embeddings, and task-specific RNN hidden unit), or embedding-free units such as image pixels.
Open Domain Question Answering Using Early Fusion of Knowledge Bases and Text
In this paper we look at a more practical setting, namely QA over the combination of a KB and entity-linked text, which is appropriate when an incomplete KB is available with a large text corpus.
Graph Representation Learning
Open-Domain Question Answering
HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering
Existing question answering (QA) datasets fail to train QA systems to perform complex reasoning and provide explanations for answers.
Ranked #34 on
Question Answering
on HotpotQA
GLoMo: Unsupervised Learning of Transferable Relational Graphs
We also show that the learned graphs are generic enough to be transferred to different embeddings on which the graphs have not been trained (including GloVe embeddings, ELMo embeddings, and task-specific RNN hidden units), or embedding-free units such as image pixels.
Incremental Reading for Question Answering
Current state-of-the-art question answering models reason over an entire passage, not incrementally.
Probing Biomedical Embeddings from Language Models
For this we use the pre-trained LMs as fixed feature extractors and restrict the downstream task models to not have additional sequence modeling layers.
PullNet: Open Domain Question Answering with Iterative Retrieval on Knowledge Bases and Text
We focus on a setting in which a corpus is supplemented with a large but incomplete KB, and on questions that require non-trivial (e. g., ``multi-hop'') reasoning.
Transformer-XL: Language Modeling with Longer-Term Dependency
Moreover, Transformer-XL is up to 1, 800+ times faster than vanilla Transformer during evaluation.
Neural Query Language: A Knowledge Base Query Language for Tensorflow
Large knowledge bases (KBs) are useful for many AI tasks, but are difficult to integrate into modern gradient-based learning systems.
Differentiable Representations For Multihop Inference Rules
We present efficient differentiable implementations of second-order multi-hop reasoning using a large symbolic knowledge base (KB).
Handling Divergent Reference Texts when Evaluating Table-to-Text Generation
Automatically constructed datasets for generating text from semi-structured data (tables), such as WikiBio, often contain reference texts that diverge from the information in the corresponding semi-structured data.
PubMedQA: A Dataset for Biomedical Research Question Answering
We introduce PubMedQA, a novel biomedical question answering (QA) dataset collected from PubMed abstracts.
Ranked #7 on
Question Answering
on PubMedQA
Game Design for Eliciting Distinguishable Behavior
The ability to inferring latent psychological traits from human behavior is key to developing personalized human-interacting machine learning systems.
Scalable Neural Methods for Reasoning With a Symbolic Knowledge Base
We describe a novel way of representing a symbolic knowledge base (KB) called a sparse-matrix reified KB.
Differentiable Reasoning over a Virtual Knowledge Base
In particular, we describe a neural module, DrKIT, that traverses textual data like a KB, softly following paths of relations between mentions of entities in the corpus.
Faithful Embeddings for Knowledge Base Queries
We address this problem with a novel QE method that is more faithful to deductive reasoning, and show that this leads to better performance on complex queries to incomplete KBs.
Facts as Experts: Adaptable and Interpretable Neural Memory over Symbolic Knowledge
Massive language models are the core of modern NLP modeling and have been shown to encode impressive amounts of commonsense and factual information.
Open Question Answering over Tables and Text
In open question answering (QA), the answer to a question is produced by retrieving and then analyzing documents that might contain answers to the question.
Ranked #1 on
Question Answering
on OTT-QA
Differentiable Open-Ended Commonsense Reasoning
As a step towards making commonsense reasoning research more realistic, we propose to study open-ended commonsense reasoning (OpenCSR) -- the task of answering a commonsense question without any pre-defined choices -- using as a resource only a corpus of commonsense facts written in natural language.
Evaluating Explanations: How much do explanations from the teacher aid students?
While many methods purport to explain predictions by highlighting salient features, what aims these explanations serve and how they ought to be evaluated often go unstated.
Reasoning Over Virtual Knowledge Bases With Open Predicate Relations
We present the Open Predicate Query Language (OPQL); a method for constructing a virtual KB (VKB) trained entirely from text.
What's the best place for an AI conference, Vancouver or ______: Why completing comparative questions is difficult
Here we study using such LMs to fill in entities in human-authored comparative questions, like ``Which country is older, India or ______?''
Iterative Hierarchical Attention for Answering Complex Questions over Long Documents
We propose a new model, DocHopper, that iteratively attends to different parts of long, hierarchically structured documents to answer complex questions.
Ranked #2 on
Question Answering
on ConditionalQA
Time-Aware Language Models as Temporal Knowledge Bases
We introduce a diagnostic dataset aimed at probing LMs for factual knowledge that changes over time and highlight problems with LMs at either end of the spectrum -- those trained on specific slices of temporal data, as well as those trained on a wide range of temporal data.
MATE: Multi-view Attention for Table Transformer Efficiency
However, more than 20% of relational tables on the web have 20 or more rows (Cafarella et al., 2008), and these large tables present a challenge for current Transformer models, which are typically limited to 512 tokens.
Ranked #2 on
Question Answering
on HybridQA
Multilingual Fact Linking
This makes it challenging to link KG facts to sentences in languages other than the limited set of languages.
ConditionalQA: A Complex Reading Comprehension Dataset with Conditional Answers
In addition to conditional answers, the dataset also features: (1) long context documents with information that is related in logically complex ways; (2) multi-hop questions that require compositional logical reasoning; (3) a combination of extractive questions, yes/no questions, questions with multiple answers, and not-answerable questions; (4) questions asked without knowing the answers.
Explain, Edit, and Understand: Rethinking User Study Design for Evaluating Model Explanations
Through our evaluation, we observe that for a linear bag-of-words model, participants with access to the feature coefficients during training are able to cause a larger reduction in model confidence in the testing phase when compared to the no-explanation control.
Transformer Memory as a Differentiable Search Index
In this paper, we demonstrate that information retrieval can be accomplished with a single Transformer, in which all information about the corpus is encoded in the parameters of the model.
Reasoning over Logically Interacted Conditions for Question Answering
Even more challenging, we only provide evidences for a subset of the conditions, so some questions may not have deterministic answers.
QA Is the New KR: Question-Answer Pairs as Knowledge Bases
In this position paper, we propose a new approach to generating a type of knowledge base (KB) from text, based on question generation and entity linking.
WinoDict: Probing language models for in-context word acquisition
We introduce a new in-context learning paradigm to measure Large Language Models' (LLMs) ability to learn novel words during inference.
Re-Imagen: Retrieval-Augmented Text-to-Image Generator
To further evaluate the capabilities of the model, we introduce EntityDrawBench, a new benchmark that evaluates image generation for diverse entities, from frequent to rare, across multiple object categories including dogs, foods, landmarks, birds, and characters.
Ranked #3 on
Text-to-Image Generation
on MS COCO
MuRAG: Multimodal Retrieval-Augmented Generator for Open Question Answering over Images and Text
While language Models store a massive amount of world knowledge implicitly in their parameters, even very large models often fail to encode information about rare entities and events, while incurring huge computational costs.
Correcting Diverse Factual Errors in Abstractive Summarization via Post-Editing and Language Model Infilling
Abstractive summarization models often generate inconsistent summaries containing factual errors or hallucinated content.
Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Tasks
By combining PoT with self-consistency decoding, we can achieve SoTA performance on all math problem datasets and near-SoTA performance on financial datasets.
Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models
We take human annotations as a gold standard and show that a correlated automatic metric is suitable for development.
Beyond Contrastive Learning: A Variational Generative Model for Multilingual Retrieval
Contrastive learning has been successfully used for retrieval of semantically aligned sentences, but it often requires large batch sizes or careful engineering to work well.
GLIMMER: generalized late-interaction memory reranker
Memory-augmentation is a powerful approach for efficiently incorporating external information into language models, but leads to reduced performance relative to retrieving text.
Answering Ambiguous Questions with a Database of Questions, Answers, and Revisions
Many open-domain questions are under-specified and thus have multiple possible answers, each of which is correct under a different interpretation of the question.
MEMORY-VQ: Compression for Tractable Internet-Scale Memory
Retrieval augmentation is a powerful but expensive method to make language models more knowledgeable about the world.
SEMQA: Semi-Extractive Multi-Source Question Answering
Experimenting with several LLMs in various settings, we find this task to be surprisingly challenging, demonstrating the importance of QuoteSum for developing and studying such consolidation capabilities.
Trusted Source Alignment in Large Language Models
Large language models (LLMs) are trained on web-scale corpora that inevitably include contradictory factual information from sources of varying reliability.
Characterizing Tradeoffs in Language Model Decoding with Informational Interpretations
We propose a theoretical framework for formulating language model decoder algorithms with dynamic programming and information theory.
