Search Results for author:
Found 157 papers, 67 papers with code
Training Factor Graphs with Reinforcement Learning for Efficient MAP Inference
Large, relational factor graphs with structure defined by first-order logic or other languages give rise to notoriously difficult inference problems.
FACTORIE: Probabilistic Programming via Imperatively Defined Factor Graphs
Discriminatively trained undirected graphical models have had wide empirical success, and there has been increasing interest in toolkits that ease their application to complex relational data.
Rethinking LDA: Why Priors Matter
Implementations of topic models typically use symmetric Dirichlet priors with fixed concentration parameters, with the implicit assumption that such smoothing parameters" have little practical effect.
An Introduction to Conditional Random Fields
This tutorial describes conditional random fields, a popular probabilistic method for structured prediction.
Query-Aware MCMC
Traditional approaches to probabilistic inference such as loopy belief propagation and Gibbs sampling typically compute marginals for it all the unobserved variables in a graphical model.
MAP Inference in Chains using Column Generation
Linear chains and trees are basic building blocks in many applications of graphical models.
Anytime Belief Propagation Using Sparse Domains
Belief Propagation has been widely used for marginal inference, however it is slow on problems with large-domain variables and high-order factors.
Learning Soft Linear Constraints with Application to Citation Field Extraction
Accurately segmenting a citation string into fields for authors, titles, etc.
Lexicon Infused Phrase Embeddings for Named Entity Resolution
Most state-of-the-art approaches for named-entity recognition (NER) use semi supervised information in the form of word clusters and lexicons.
Training for Fast Sequential Prediction Using Dynamic Feature Selection
We present paired learning and inference algorithms for significantly reducing computation and increasing speed of the vector dot products in the classifiers that are at the heart of many NLP components.
Word Representations via Gaussian Embedding
Current work in lexical distributed representations maps each word to a point vector in low-dimensional space.
Bethe Projections for Non-Local Inference
Many inference problems in structured prediction are naturally solved by augmenting a tractable dependency structure with complex, non-local auxiliary objectives.
Efficient Non-parametric Estimation of Multiple Embeddings per Word in Vector Space
There is rising interest in vector-space word embeddings and their use in NLP, especially given recent methods for their fast estimation at very large scale.
Compositional Vector Space Models for Knowledge Base Completion
Knowledge base (KB) completion adds new facts to a KB by making inferences from existing facts, for example by inferring with high likelihood nationality(X, Y) from bornIn(X, Y).
Learning Dynamic Feature Selection for Fast Sequential Prediction
We present paired learning and inference algorithms for significantly reducing computation and increasing speed of the vector dot products in the classifiers that are at the heart of many NLP components.
Multilingual Relation Extraction using Compositional Universal Schema
In response, this paper introduces significant further improvements to the coverage and flexibility of universal schema relation extraction: predictions for entities unseen in training and multilingual transfer learning to domains with no annotation.
Structured Prediction Energy Networks
This deep architecture captures dependencies between labels that would lead to intractable graphical models, and performs structure learning by automatically learning discriminative features of the structured output.
Row-less Universal Schema
In experimental results on the FB15k-237 benchmark we demonstrate that we can match the performance of a comparable model with explicit entity pair representations using a model of attention over relation types.
Generalizing to Unseen Entities and Entity Pairs with Row-less Universal Schema
In experiments predicting both relations and entity types, we demonstrate that despite having an order of magnitude fewer parameters than traditional universal schema, we can match the accuracy of the traditional model, and more importantly, we can now make predictions about unseen rows with nearly the same accuracy as rows available at training time.
Chains of Reasoning over Entities, Relations, and Text using Recurrent Neural Networks
Our goal is to combine the rich multistep inference of symbolic logical reasoning with the generalization capabilities of neural networks.
Ask the GRU: Multi-Task Learning for Deep Text Recommendations
In a variety of application domains the content to be recommended to users is associated with text.
Learning a Natural Language Interface with Neural Programmer
The main experimental result in this paper is that a single Neural Programmer model achieves 34. 2% accuracy using only 10, 000 examples with weak supervision.
Fast and Accurate Entity Recognition with Iterated Dilated Convolutions
Today when many practitioners run basic NLP on the entire web and large-volume traffic, faster methods are paramount to saving time and energy costs.
Ranked #24 on
Named Entity Recognition (NER)
on Ontonotes v5 (English)
End-to-End Learning for Structured Prediction Energy Networks
Structured Prediction Energy Networks (SPENs) are a simple, yet expressive family of structured prediction models (Belanger and McCallum, 2016).
An Online Hierarchical Algorithm for Extreme Clustering
Many modern clustering methods scale well to a large number of data items, N, but not to a large number of clusters, K. This paper introduces PERCH, a new non-greedy algorithm for online hierarchical clustering that scales to both massive N and K--a problem setting we term extreme clustering.
SemEval 2017 Task 10: ScienceIE - Extracting Keyphrases and Relations from Scientific Publications
We describe the SemEval task of extracting keyphrases and relations between them from scientific documents, which is crucial for understanding which publications describe which processes, tasks and materials.
Active Bias: Training More Accurate Neural Networks by Emphasizing High Variance Samples
Self-paced learning and hard example mining re-weight training instances to improve learning accuracy.
Question Answering on Knowledge Bases and Text using Universal Schema and Memory Networks
Existing question answering methods infer answers either from a knowledge base or from raw text.
Dependency Parsing with Dilated Iterated Graph CNNs
Dependency parses are an effective way to inject linguistic knowledge into many downstream tasks, and many practitioners wish to efficiently parse sentences at scale.
RelNet: End-to-End Modeling of Entities & Relations
We introduce RelNet: a new model for relational reasoning.
Improved Representation Learning for Predicting Commonsense Ontologies
Recent work in learning ontologies (hierarchical and partially-ordered structures) has leveraged the intrinsic geometry of spaces of learned representations to make predictions that automatically obey complex structural constraints.
Low-Rank Hidden State Embeddings for Viterbi Sequence Labeling
In textual information extraction and other sequence labeling tasks it is now common to use recurrent neural networks (such as LSTM) to form rich embedded representations of long-term input co-occurrence patterns.
Distributional Inclusion Vector Embedding for Unsupervised Hypernymy Detection
Modeling hypernymy, such as poodle is-a dog, is an important generalization aid to many NLP tasks, such as entailment, coreference, relation extraction, and question answering.
Attending to All Mention Pairs for Full Abstract Biological Relation Extraction
We propose a model to consider all mention and entity pairs simultaneously in order to make a prediction.
Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning
Knowledge bases (KB), both automatically and manually constructed, are often incomplete --- many valid facts can be inferred from the KB by synthesizing existing information.
Finer Grained Entity Typing with TypeNet
We consider the challenging problem of entity typing over an extremely fine grained set of types, wherein a single mention or entity can have many simultaneous and often hierarchically-structured types.
Automatically Extracting Action Graphs from Materials Science Synthesis Procedures
In this work, we present a system for automatically extracting structured representations of synthesis procedures from the texts of materials science journal articles that describe explicit, experimental syntheses of inorganic compounds.
Simultaneously Self-Attending to All Mentions for Full-Abstract Biological Relation Extraction
Most work in relation extraction forms a prediction by looking at a short span of text within a single sentence containing a single entity pair mention.
Efficient Graph-based Word Sense Induction by Distributional Inclusion Vector Embeddings
Word sense induction (WSI), which addresses polysemy by unsupervised discovery of multiple word senses, resolves ambiguities for downstream NLP tasks and also makes word representations more interpretable.
Linguistically-Informed Self-Attention for Semantic Role Labeling
Unlike previous models which require significant pre-processing to prepare linguistic features, LISA can incorporate syntax using merely raw tokens as input, encoding the sequence only once to simultaneously perform parsing, predicate detection and role labeling for all predicates.
Probabilistic Embedding of Knowledge Graphs with Box Lattice Measures
Embedding methods which enforce a partial order or lattice structure over the concept space, such as Order Embeddings (OE) (Vendrov et al., 2016), are a natural way to model transitive relational data (e. g. entailment graphs).
A Systematic Classification of Knowledge, Reasoning, and Context within the ARC Dataset
We propose a comprehensive set of definitions of knowledge and reasoning types necessary for answering the questions in the ARC dataset.
Training Structured Prediction Energy Networks with Indirect Supervision
This paper introduces rank-based training of structured prediction energy networks (SPENs).
Hierarchical Losses and New Resources for Fine-grained Entity Typing and Linking
Extraction from raw text to a knowledge base of entities and fine-grained types is often cast as prediction into a flat set of entity and type labels, neglecting the rich hierarchies over types and entities contained in curated ontologies.
Embedded-State Latent Conditional Random Fields for Sequence Labeling
Complex textual information extraction tasks are often posed as sequence labeling or \emph{shallow parsing}, where fields are extracted using local labels made consistent through probabilistic inference in a graphical model with constrained transitions.
Marginal Likelihood Training of BiLSTM-CRF for Biomedical Named Entity Recognition from Disjoint Label Sets
This paper presents a method for training a single CRF extractor from multiple datasets with disjoint or partially overlapping sets of entity types.
Building Dynamic Knowledge Graphs from Text using Machine Reading Comprehension
We harness and extend a recently proposed machine reading comprehension (MRC) model to query for entity states, since these states are generally communicated in spans of text and MRC models perform well in extracting entity-centric spans.
Ranked #3 on
Procedural Text Understanding
on ProPara
An Interface for Annotating Science Questions
Recent work introduces the AI2 Reasoning Challenge (ARC) and the associated ARC dataset that partitions open domain, complex science questions into an Easy Set and a Challenge Set.
Syntax Helps ELMo Understand Semantics: Is Syntax Still Relevant in a Deep Neural Architecture for SRL?
Do unsupervised methods for learning rich, contextualized token representations obviate the need for explicit modeling of linguistic structure in neural network models for semantic role labeling (SRL)?
Integrating User Feedback under Identity Uncertainty in Knowledge Base Construction
Users have tremendous potential to aid in the construction and maintenance of knowledges bases (KBs) through the contribution of feedback that identifies incorrect and missing entity attributes and relations.
Compact Representation of Uncertainty in Clustering
For many classic structured prediction problems, probability distributions over the dependent variables can be efficiently computed using widely-known algorithms and data structures (such as forward-backward, and its corresponding trellis for exact probability distributions in Markov models).
Search-Guided, Lightly-supervised Training of Structured Prediction Energy Networks
In this paper, we instead use efficient truncated randomized search in this reward function to train structured prediction energy networks (SPENs), which provide efficient test-time inference using gradient-based search on a smooth, learned representation of the score landscape, and have previously yielded state-of-the-art results in structured prediction.
Inorganic Materials Synthesis Planning with Literature-Trained Neural Networks
Leveraging new data sources is a key step in accelerating the pace of materials design and discovery.
Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Autoencoders
We introduce deep inside-outside recursive autoencoders (DIORA), a fully-unsupervised method for discovering syntax that simultaneously learns representations for constituents within the induced tree.
OpenKI: Integrating Open Information Extraction and Knowledge Bases with Relation Inference
In this paper, we consider advancing web-scale knowledge extraction and alignment by integrating OpenIE extractions in the form of (subject, predicate, object) triples with Knowledge Bases (KB).
Smoothing the Geometry of Probabilistic Box Embeddings
However, the hard edges of the boxes present difficulties for standard gradient based optimization; that work employed a special surrogate function for the disjoint case, but we find this method to be fragile.
Multi-step Retriever-Reader Interaction for Scalable Open-domain Question Answering
This paper introduces a new framework for open-domain question answering in which the retriever and the reader iteratively interact with each other.
The Materials Science Procedural Text Corpus: Annotating Materials Synthesis Procedures with Shallow Semantic Structures
Materials science literature contains millions of materials synthesis procedures described in unstructured natural language text.
Paper Matching with Local Fairness Constraints
Automatically matching reviewers to papers is a crucial step of the peer review process for venues receiving thousands of submissions.
Data Structures and Algorithms Digital Libraries
Unsupervised Latent Tree Induction with Deep Inside-Outside Recursive Auto-Encoders
We introduce the deep inside-outside recursive autoencoder (DIORA), a fully-unsupervised method for discovering syntax that simultaneously learns representations for constituents within the induced tree.
Energy and Policy Considerations for Deep Learning in NLP
Recent progress in hardware and methodology for training neural networks has ushered in a new generation of large networks trained on abundant data.
Supervised Hierarchical Clustering with Exponential Linkage
Thus we introduce an approach to supervised hierarchical clustering that smoothly interpolates between single, average, and complete linkage, and we give a training procedure that simultaneously learns a linkage function and a dissimilarity function.
A2N: Attending to Neighbors for Knowledge Graph Inference
State-of-the-art models for knowledge graph completion aim at learning a fixed embedding representation of entities in a multi-relational graph which can generalize to infer unseen entity relationships at test time.
Optimal Transport-based Alignment of Learned Character Representations for String Similarity
We evaluate STANCE's ability to detect whether two strings can refer to the same entity--a task we term alias detection.
Multi-step Entity-centric Information Retrieval for Multi-Hop Question Answering
Multi-hop question answering (QA) requires an information retrieval (IR) system that can find \emph{multiple} supporting evidence needed to answer the question, making the retrieval process very challenging.
Roll Call Vote Prediction with Knowledge Augmented Models
The sources of knowledge we use are news text and Freebase, a manually curated knowledge base.
Unsupervised Labeled Parsing with Deep Inside-Outside Recursive Autoencoders
Understanding text often requires identifying meaningful constituent spans such as noun phrases and verb phrases.
Chains-of-Reasoning at TextGraphs 2019 Shared Task: Reasoning over Chains of Facts for Explainable Multi-hop Inference
This paper describes our submission to the shared task on {``}Multi-hop Inference Explanation Regeneration{''} in TextGraphs workshop at EMNLP 2019 (Jansen and Ustalov, 2019).
Learning to Few-Shot Learn Across Diverse Natural Language Classification Tasks
LEOPARD is trained with the state-of-the-art transformer architecture and shows better generalization to tasks not seen at all during training, with as few as 4 examples per label.
Using Error Decay Prediction to Overcome Practical Issues of Deep Active Learning for Named Entity Recognition
Existing deep active learning algorithms achieve impressive sampling efficiency on natural language processing tasks.
Simultaneously Linking Entities and Extracting Relations from Biomedical Text Without Mention-level Supervision
Understanding the meaning of text often involves reasoning about entities and their relationships.
Scalable Hierarchical Clustering with Tree Grafting
We introduce Grinch, a new algorithm for large-scale, non-greedy hierarchical clustering with general linkage functions that compute arbitrary similarity between two point sets.
Representing Joint Hierarchies with Box Embeddings
Box Embeddings [Vilnis et al., 2018, Li et al., 2019] represent concepts with hyperrectangles in $n$-dimensional space and are shown to be capable of modeling tree-like structures efficiently by training on a large subset of the transitive closure of the WordNet hypernym graph.
Predicting Institution Hierarchies with Set-based Models
The hierarchical structure of research organizations plays a pivotal role in science of science research as well as in tools that track the research achievements and output.
Data Structures & Algorithms for Exact Inference in Hierarchical Clustering
In contrast to existing methods, we present novel dynamic-programming algorithms for \emph{exact} inference in hierarchical clustering based on a novel trellis data structure, and we prove that we can exactly compute the partition function, maximum likelihood hierarchy, and marginal probabilities of sub-hierarchies and clusters.
ProtoQA: A Question Answering Dataset for Prototypical Common-Sense Reasoning
Given questions regarding some prototypical situation such as Name something that people usually do before they leave the house for work?
Using BibTeX to Automatically Generate Labeled Data for Citation Field Extraction
In this paper, we describe a technique for using BibTeX to generate, automatically, a large-scale 41M labeled strings), labeled dataset, that is four orders of magnitude larger than the current largest CFE dataset, namely the UMass Citation Field Extraction dataset [Anzaroot and McCallum, 2013].
AutoKnow: Self-Driving Knowledge Collection for Products of Thousands of Types
Can one build a knowledge graph (KG) for all products in the world?
A Simple Approach to Case-Based Reasoning in Knowledge Bases
We present a surprisingly simple yet accurate approach to reasoning in knowledge graphs (KGs) that requires \emph{no training}, and is reminiscent of case-based reasoning in classical artificial intelligence (AI).
Self-Supervised Meta-Learning for Few-Shot Natural Language Classification Tasks
We meta-train a transformer model on this distribution of tasks using a recent meta-learning framework.
Energy-Based Reranking: Improving Neural Machine Translation Using Energy-Based Models
To benefit from this observation, we train an energy-based model to mimic the behavior of the task measure (i. e., the energy-based model assigns lower energy to samples with higher BLEU score), which is resulted in a re-ranking algorithm based on the samples drawn from NMT: energy-based re-ranking (EBR).
Unsupervised Pre-training for Biomedical Question Answering
We explore the suitability of unsupervised representation learning methods on biomedical text -- BioBERT, SciBERT, and BioSentVec -- for biomedical question answering.
Probabilistic Case-based Reasoning for Open-World Knowledge Graph Completion
A case-based reasoning (CBR) system solves a new problem by retrieving `cases' that are similar to the given problem.
Ranked #1 on
Link Prediction
on NELL-995
Improving Local Identifiability in Probabilistic Box Embeddings
Geometric embeddings have recently received attention for their natural ability to represent transitive asymmetric relations via containment.
Clustering-based Inference for Biomedical Entity Linking
In this paper, we introduce a model in which linking decisions can be made not merely by linking to a knowledge base entity but also by grouping multiple mentions together via clustering and jointly making linking predictions.
Scalable Hierarchical Agglomerative Clustering
The applicability of agglomerative clustering, for inferring both hierarchical and flat clustering, is limited by its scalability.
An Instance Level Approach for Shallow Semantic Parsing in Scientific Procedural Text
Fortunately, such specific domains often use rather formulaic writing, such that the different ways of expressing relations in a small number of grammatically similar labeled sentences may provide high coverage of semantic structures in the corpus, through an appropriately rich similarity metric.
Box-To-Box Transformation for Modeling Joint Hierarchies
In Patel et al. (2020), the authors demonstrate that only the transitive reduction is required, and further extend box embeddings to capture joint hierarchies by augmenting the graph with new nodes.
Modeling Fine-Grained Entity Types with Box Embeddings
Neural entity typing models typically represent fine-grained entity types as vectors in a high-dimensional space, but such spaces are not well-suited to modeling these types' complex interdependencies.
Ranked #9 on
Entity Typing
on Open Entity
Low Resource Recognition and Linking of Biomedical Concepts from a Large Ontology
Tools to explore scientific literature are essential for scientists, especially in biomedicine, where about a million new papers are published every year.
Long Document Summarization in a Low Resource Setting using Pretrained Language Models
Abstractive summarization is the task of compressing a long document into a coherent short document while retaining salient information.
CSFCube -- A Test Collection of Computer Science Research Articles for Faceted Query by Example
Query by Example is a well-known information retrieval task in which a document is chosen by the user as the search query and the goal is to retrieve relevant documents from a large collection.
Multi-facet Universal Schema
To address the violation of the USchema assumption, we propose multi-facet universal schema that uses a neural model to represent each sentence pattern as multiple facet embeddings and encourage one of these facet embeddings to be close to that of another sentence pattern if they co-occur with the same entity pair.
Changing the Mind of Transformers for Topically-Controllable Language Generation
Our framework consists of two components: (1) a method that produces a set of candidate topics by predicting the centers of word clusters in the possible continuations, and (2) a text generation model whose output adheres to the chosen topics.
Extending Multi-Sense Word Embedding to Phrases and Sentences for Unsupervised Semantic Applications
Most unsupervised NLP models represent each word with a single point or single region in semantic space, while the existing multi-sense word embeddings cannot represent longer word sequences like phrases or sentences.
Probabilistic Box Embeddings for Uncertain Knowledge Graph Reasoning
Knowledge bases often consist of facts which are harvested from a variety of sources, many of which are noisy and some of which conflict, resulting in a level of uncertainty for each triple.
Exact and Approximate Hierarchical Clustering Using A*
In those cases, hierarchical clustering can be seen as a combinatorial optimization problem.
Case-based Reasoning for Natural Language Queries over Knowledge Bases
It is often challenging to solve a complex problem from scratch, but much easier if we can access other similar problems with their solutions -- a paradigm known as case-based reasoning (CBR).
Knowledge Base Question Answering
Natural Language Queries
+1
MOLEMAN: Mention-Only Linking of Entities with a Mention Annotation Network
We present an instance-based nearest neighbor approach to entity linking.
Word2Box: Capturing Set-Theoretic Semantics of Words using Box Embeddings
In this work, we provide a fuzzy-set interpretation of box embeddings, and learn box representations of words using a set-theoretic training objective.
Benchmarking Scalable Methods for Streaming Cross Document Entity Coreference
We investigate: how to best encode mentions, which clustering algorithms are most effective for grouping mentions, how models transfer to different domains, and how bounding the number of mentions tracked during inference impacts performance.
Entity Linking and Discovery via Arborescence-based Supervised Clustering
Previous work has shown promising results in performing entity linking by measuring not only the affinities between mentions and entities but also those amongst mentions.
Box Embeddings: An open-source library for representation learning using geometric structures
A major factor contributing to the success of modern representation learning is the ease of performing various vector operations.
Improved Latent Tree Induction with Distant Supervision via Span Constraints
For over thirty years, researchers have developed and analyzed methods for latent tree induction as an approach for unsupervised syntactic parsing.
Structured Energy Network as a dynamic loss function. Case study. A case study with multi-label Classification
The best version of SEAL that uses NCE ranking method achieves close to +2. 85, +2. 23 respective F1 point gain in average over cross-entropy and INFNET on the feature-based datasets, excluding one outlier that has an excessive gain of +50. 0 F1 points.
DISAPERE: A Dataset for Discourse Structure in Peer Review Discussions
At the foundation of scientific evaluation is the labor-intensive process of peer review.
Diverse Distributions of Self-Supervised Tasks for Meta-Learning in NLP
Meta-learning considers the problem of learning an efficient learning process that can leverage its past experience to accurately solve new tasks.
Capacity and Bias of Learned Geometric Embeddings for Directed Graphs
While vectors in Euclidean space can theoretically represent any graph, much recent work shows that alternatives such as complex, hyperbolic, order, or box embeddings have geometric properties better suited to modeling real-world graphs.
Sublinear Time Approximation of Text Similarity Matrices
Approximation methods reduce this quadratic complexity, often by using a small subset of exactly computed similarities to approximate the remainder of the complete pairwise similarity matrix.
Knowledge Base Question Answering by Case-based Reasoning over Subgraphs
Question answering (QA) over knowledge bases (KBs) is challenging because of the diverse, essentially unbounded, types of reasoning patterns needed.
A Distant Supervision Corpus for Extracting Biomedical Relationships Between Chemicals, Diseases and Genes
We introduce ChemDisGene, a new dataset for training and evaluating multi-class multi-label document-level biomedical relation extraction models.
CBR-iKB: A Case-Based Reasoning Approach for Question Answering over Incomplete Knowledge Bases
Yet, in many question answering applications coupled with knowledge bases, the sparse nature of KBs is often overlooked.
Inducing and Using Alignments for Transition-based AMR Parsing
These alignments are learned separately from parser training and require a complex pipeline of rule-based components, pre-processing, and post-processing to satisfy domain-specific constraints.
Augmenting Scientific Creativity with Retrieval across Knowledge Domains
Exposure to ideas in domains outside a scientist's own may benefit her in reformulating existing research problems in novel ways and discovering new application domains for existing solution ideas.
Meta-Adapters: Parameter Efficient Few-shot Fine-tuning through Meta-Learning
Meta-Adapters perform competitively with state-of-the-art few-shot learning methods that require full fine-tuning, while only fine-tuning 0. 6% of the parameters.
Longtonotes: OntoNotes with Longer Coreference Chains
Ontonotes has served as the most important benchmark for coreference resolution.
Multi-CLS BERT: An Efficient Alternative to Traditional Ensembling
Ensembling BERT models often significantly improves accuracy, but at the cost of significantly more computation and memory footprint.
Efficient Nearest Neighbor Search for Cross-Encoder Models using Matrix Factorization
When the similarity is measured by dot-product between dual-encoder vectors or $\ell_2$-distance, there already exist many scalable and efficient search methods.
You can't pick your neighbors, or can you? When and how to rely on retrieval in the $k$NN-LM
Retrieval-enhanced language models (LMs), which condition their predictions on text retrieved from large external datastores, have recently shown significant perplexity improvements compared to standard LMs.
Ranked #9 on
Language Modelling
on WikiText-103
Low-Resource Compositional Semantic Parsing with Concept Pretraining
In this work, we present an architecture to perform such domain adaptation automatically, with only a small amount of metadata about the new domain and without any new training data (zero-shot) or with very few examples (few-shot).
Improving Dual-Encoder Training through Dynamic Indexes for Negative Mining
First, we introduce an algorithm that uses a tree structure to approximate the softmax with provable bounds and that dynamically maintains the tree.
Editable User Profiles for Controllable Text Recommendation
Finally, we implement LACE in an interactive controllable recommender system and conduct a user study to demonstrate that users are able to improve the quality of recommendations they receive through interactions with an editable user profile.
Efficient k-NN Search with Cross-Encoders using Adaptive Multi-Round CUR Decomposition
While ANNCUR's one-time selection of anchors tends to approximate the cross-encoder distances on average, doing so forfeits the capacity to accurately estimate distances to items near the query, leading to regret in the crucial end-task: recall of top-k items.
Revisiting the Architectures like Pointer Networks to Efficiently Improve the Next Word Distribution, Summarization Factuality, and Beyond
Is the output softmax layer, which is adopted by most language models (LMs), always the best way to compute the next word probability?
Machine Reading Comprehension using Case-based Reasoning
Given a test question, CBR-MRC first retrieves a set of similar cases from a nonparametric memory and then predicts an answer by selecting the span in the test context that is most similar to the contextualized representations of answers in the retrieved cases.
Large Language Model Augmented Narrative Driven Recommendations
Narrative-driven recommendation (NDR) presents an information access problem where users solicit recommendations with verbose descriptions of their preferences and context, for example, travelers soliciting recommendations for points of interest while describing their likes/dislikes and travel circumstances.
Answering Compositional Queries with Set-Theoretic Embeddings
The need to compactly and robustly represent item-attribute relations arises in many important tasks, such as faceted browsing and recommendation systems.
Encoding Multi-Domain Scientific Papers by Ensembling Multiple CLS Tokens
Many useful tasks on scientific documents, such as topic classification and citation prediction, involve corpora that span multiple scientific domains.
To Copy, or not to Copy; That is a Critical Issue of the Output Softmax Layer in Neural Sequential Recommenders
Specifically, the similarity structure of the global item embeddings in the softmax layer sometimes forces the single hidden state embedding to be close to new items when copying is a better choice, while sometimes forcing the hidden state to be close to the items from the input inappropriately.
PaRaDe: Passage Ranking using Demonstrations with Large Language Models
Recent studies show that large language models (LLMs) can be instructed to effectively perform zero-shot passage re-ranking, in which the results of a first stage retrieval method, such as BM25, are rated and reordered to improve relevance.
Multistage Collaborative Knowledge Distillation from a Large Language Model for Semi-Supervised Sequence Generation
In this paper, we present the discovery that a student model distilled from a few-shot prompted LLM can commonly generalize better than its teacher to unseen examples on such tasks.
Fast, Scalable, Warm-Start Semidefinite Programming with Spectral Bundling and Sketching
We present Unified Spectral Bundling with Sketching (USBS), a provably correct, fast and scalable algorithm for solving massive SDPs that can leverage a warm-start initialization to further accelerate convergence.
Incremental Extractive Opinion Summarization Using Cover Trees
In this work, we study the task of extractive opinion summarization in an incremental setting, where the underlying review set evolves over time.
Event-Event Relation Extraction using Probabilistic Box Embedding
To understand a story with multiple events, it is important to capture the proper relations across these events.
Softmax Bottleneck Makes Language Models Unable to Represent Multi-mode Word Distributions
The softmax layer produces the distribution based on the dot products of a single hidden state and the embeddings of words in the vocabulary.
Unsupervised Parsing with S-DIORA: Single Tree Encoding for Deep Inside-Outside Recursive Autoencoders
The deep inside-outside recursive autoencoder (DIORA; Drozdov et al. 2019) is a self-supervised neural model that learns to induce syntactic tree structures for input sentences *without access to labeled training data*.
Box-To-Box Transformations for Modeling Joint Hierarchies
In Patel et al., (2020), the authors demonstrate that only the transitive reduction is required and further extend box embeddings to capture joint hierarchies by augmenting the graph with new nodes.
Simultaneously Self-Attending to Text and Entities for Knowledge-Informed Text Representations
In this work, we aim at directly learning text representations which leverage structured knowledge about entities mentioned in the text.
Knowledge Informed Semantic Parsing for Conversational Question Answering
Specially, neural semantic parsers (NSPs) effectively translate natural questions to logical forms, which execute on KB and give desirable answers.
Event and Entity Coreference using Trees to Encode Uncertainty in Joint Decisions
Coreference decisions among event mentions and among co-occurring entity mentions are highly interdependent, thus motivating joint inference.
Enhanced Distant Supervision with State-Change Information for Relation Extraction
In this work, we introduce a method for enhancing distant supervision with state-change information for relation extraction.
Entity Linking via Explicit Mention-Mention Coreference Modeling
Learning representations of entity mentions is a core component of modern entity linking systems for both candidate generation and making linking predictions.
Ranked #1 on
Entity Linking
on ZESHEL
Unsupervised Partial Sentence Matching for Cited Text Identification
Given a citation in the body of a research paper, cited text identification aims to find the sentences in the cited paper that are most relevant to the citing sentence.
MS-Mentions: Consistently Annotating Entity Mentions in Materials Science Procedural Text
Material science synthesis procedures are a promising domain for scientific NLP, as proper modeling of these recipes could provide insight into new ways of creating materials.
