Search Results for author:
Found 60 papers, 34 papers with code
Definition Modeling: Learning to define word embeddings in natural language
Distributed representations of words have been shown to capture lexical semantics, as demonstrated by their effectiveness in word similarity and analogical relation tasks.
VecShare: A Framework for Sharing Word Representation Vectors
Many Natural Language Processing (NLP) models rely on distributed vector representations of words.
Construction of the Literature Graph in Semantic Scholar
We describe a deployed scalable system for organizing published scientific literature into a heterogeneous graph to facilitate algorithmic manipulation and discovery.
Sampling Informative Training Data for RNN Language Models
We propose an unsupervised importance sampling approach to selecting training data for recurrent neural network (RNNs) language models.
Extracting Commonsense Properties from Embeddings with Limited Human Guidance
Intelligent systems require common sense, but automatically extracting this knowledge from text can be difficult.
Estimating Marginal Probabilities of n-grams for Recurrent Neural Language Models
Recurrent neural network language models (RNNLMs) are the current standard-bearer for statistical language modeling.
A new evaluation framework for topic modeling algorithms based on synthetic corpora
Topic models are in widespread use in natural language processing and beyond.
CODAH: An Adversarially Authored Question-Answer Dataset for Common Sense
To produce a more difficult dataset, we introduce a novel procedure for question acquisition in which workers author questions designed to target weaknesses of state-of-the-art neural question answering systems.
Ranked #1 on
Common Sense Reasoning
on CODAH
(using extra training data)
Using Large Corpus N-gram Statistics to Improve Recurrent Neural Language Models
Recurrent neural network language models (RNNLM) form a valuable foundation for many NLP systems, but training the models can be computationally expensive, and may take days to train on a large corpus.
A Semantic Cover Approach for Topic Modeling
We introduce a novel topic modeling approach based on constructing a semantic set cover for clusters of similar documents.
CODAH: An Adversarially-Authored Question Answering Dataset for Common Sense
To produce a more difficult dataset, we introduce a novel procedure for question acquisition in which workers author questions designed to target weaknesses of state-of-the-art neural question answering systems.
Multi-sense Definition Modeling using Word Sense Decompositions
Word embeddings capture syntactic and semantic information about words.
Just Add Functions: A Neural-Symbolic Language Model
How can we augment today's neural models with such encodings?
LIMEADE: From AI Explanations to Advice Taking
We show our method improves accuracy compared to a rigorous baseline on the image classification domains.
SPECTER: Document-level Representation Learning using Citation-informed Transformers
We propose SPECTER, a new method to generate document-level embedding of scientific documents based on pretraining a Transformer language model on a powerful signal of document-level relatedness: the citation graph.
Ranked #1 on
Document Classification
on SciDocs (MAG)
Don't Stop Pretraining: Adapt Language Models to Domains and Tasks
Language models pretrained on text from a wide variety of sources form the foundation of today's NLP.
Generative Data Augmentation for Commonsense Reasoning
Recent advances in commonsense reasoning depend on large-scale human-annotated training data to achieve peak performance.
Ranked #1 on
Question Answering
on CODAH
Stolen Probability: A Structural Weakness of Neural Language Models
Neural Network Language Models (NNLMs) generate probability distributions by applying a softmax function to a distance metric formed by taking the dot product of a prediction vector with all word vectors in a high-dimensional embedding space.
High-Precision Extraction of Emerging Concepts from Scientific Literature
From a corpus of computer science papers on arXiv, we find that our method achieves a Precision@1000 of 99%, compared to 86% for prior work, and a substantially better precision-yield trade-off across the top 15, 000 extractions.
ABNIRML: Analyzing the Behavior of Neural IR Models
Pretrained contextualized language models such as BERT and T5 have established a new state-of-the-art for ad-hoc search.
Simplified Data Wrangling with ir_datasets
Managing the data for Information Retrieval (IR) experiments can be challenging.
SciCo: Hierarchical Cross-Document Coreference for Scientific Concepts
Determining coreference of concept mentions across multiple documents is a fundamental task in natural language understanding.
coreference-resolution
Cross Document Coreference Resolution
+1
VILA: Improving Structured Content Extraction from Scientific PDFs Using Visual Layout Groups
Experiments are conducted on a newly curated evaluation suite, S2-VLUE, that unifies existing automatically-labeled datasets and includes a new dataset of manual annotations covering diverse papers from 19 scientific disciplines.
"It doesn't look good for a date": Transforming Critiques into Preferences for Conversational Recommendation Systems
Conversations aimed at determining good recommendations are iterative in nature.
Exploring The Role of Local and Global Explanations in Recommender Systems
However, we note that the two explanation approaches may be better compared in the context of a higher-stakes or more opaque domain.
Limitations of Active Learning With Deep Transformer Language Models
Active Learning (AL) has the potential to reduce labeling cost when training natural language processing models, but its effectiveness with the large pretrained transformer language models that power today's NLP is uncertain.
Few-Shot Self-Rationalization with Natural Language Prompts
We identify the right prompting approach by extensively exploring natural language prompts on FEB. Then, by using this prompt and scaling the model size, we demonstrate that making progress on few-shot self-rationalization is possible.
Don't Say What You Don't Know: Improving the Consistency of Abstractive Summarization by Constraining Beam Search
Based on our findings, we present PINOCCHIO, a new decoding method that improves the consistency of a transformer-based abstractive summarizer by constraining beam search to avoid hallucinations.
From Who You Know to What You Read: Augmenting Scientific Recommendations with Implicit Social Networks
To improve the discovery experience we introduce multiple new methods for \em augmenting recommendations with textual relevance messages that highlight knowledge-graph connections between recommended papers and a user's publication and interaction history.
A Computational Inflection for Scientific Discovery
We stand at the foot of a significant inflection in the trajectory of scientific discovery.
ACCoRD: A Multi-Document Approach to Generating Diverse Descriptions of Scientific Concepts
We present ACCoRD, an end-to-end system tackling the novel task of generating sets of descriptions of scientific concepts.
CascadER: Cross-Modal Cascading for Knowledge Graph Link Prediction
Knowledge graph (KG) link prediction is a fundamental task in artificial intelligence, with applications in natural language processing, information retrieval, and biomedicine.
Penguins Don't Fly: Reasoning about Generics through Instantiations and Exceptions
Generics express generalizations about the world (e. g., birds can fly) that are not universally true (e. g., newborn birds and penguins cannot fly).
Multi-LexSum: Real-World Summaries of Civil Rights Lawsuits at Multiple Granularities
With the advent of large language models, methods for abstractive summarization have made great strides, creating potential for use in applications to aid knowledge workers processing unwieldy document collections.
Embedding Recycling for Language Models
Real-world applications of neural language models often involve running many different models over the same corpus.
FeedLens: Polymorphic Lenses for Personalizing Exploratory Search over Knowledge Graphs
We implement our technique in a novel system, FeedLens, which is built over Semantic Scholar, a production system for navigating the scientific literature KG.
Learning to Perform Complex Tasks through Compositional Fine-Tuning of Language Models
In this work, we present compositional fine-tuning (CFT): an approach based on explicitly decomposing a target task into component tasks, and then fine-tuning smaller LMs on a curriculum of such component tasks.
SciRepEval: A Multi-Format Benchmark for Scientific Document Representations
In response, we introduce SciRepEval, the first comprehensive benchmark for training and evaluating scientific document representations.
I2D2: Inductive Knowledge Distillation with NeuroLogic and Self-Imitation
Here, we investigate an alternative that a priori seems impossible: can smaller language models (e. g., GPT-2) win over models that are orders of magnitude larger and better (e. g., GPT-3), if powered with novel commonsense distillation algorithms?
The Semantic Scholar Open Data Platform
The volume of scientific output is creating an urgent need for automated tools to help scientists keep up with developments in their field.
Relatedly: Scaffolding Literature Reviews with Existing Related Work Sections
Scholars who want to research a scientific topic must take time to read, extract meaning, and identify connections across many papers.
The Semantic Reader Project: Augmenting Scholarly Documents through AI-Powered Interactive Reading Interfaces
Scholarly publications are key to the transfer of knowledge from scholars to others.
Beyond Summarization: Designing AI Support for Real-World Expository Writing Tasks
In this position paper, we argue that developing AI supports for expository writing has unique and exciting research challenges and can lead to high real-world impacts.
S2abEL: A Dataset for Entity Linking from Scientific Tables
Entity linking (EL) is the task of linking a textual mention to its corresponding entry in a knowledge base, and is critical for many knowledge-intensive NLP applications.
SciMON: Scientific Inspiration Machines Optimized for Novelty
We explore and enhance the ability of neural language models to generate novel scientific directions grounded in literature.
Contextualized Literature-based Discovery
Link Prediction
+1
Are Layout-Infused Language Models Robust to Layout Distribution Shifts? A Case Study with Scientific Documents
Recent work has shown that infusing layout features into language models (LMs) improves processing of visually-rich documents such as scientific papers.
ARIES: A Corpus of Scientific Paper Edits Made in Response to Peer Reviews
Revising scientific papers based on peer feedback is a challenging task that requires not only deep scientific knowledge and reasoning, but also the ability to recognize the implicit requests in high-level feedback and to choose the best of many possible ways to update the manuscript in response.
CARE: Extracting Experimental Findings From Clinical Literature
Focusing on biomedicine, this work presents CARE (Clinical Aggregation-oriented Result Extraction) -- a new IE dataset for the task of extracting clinical findings.
CHAMP: Efficient Annotation and Consolidation of Cluster Hierarchies
Various NLP tasks require a complex hierarchical structure over nodes, where each node is a cluster of items.
coreference-resolution
Cross Document Coreference Resolution
MARG: Multi-Agent Review Generation for Scientific Papers
We study the ability of LLMs to generate feedback for scientific papers and develop MARG, a feedback generation approach using multiple LLM instances that engage in internal discussion.
Who’s on First?: Probing the Learning and Representation Capabilities of Language Models on Deterministic Closed Domains
The capabilities of today’s natural language processing systems are typically evaluated using large datasets of curated questions and answers.
“It doesn’t look good for a date”: Transforming Critiques into Preferences for Conversational Recommendation Systems
Conversations aimed at determining good recommendations are iterative in nature.
