Search Results for author:
Found 21 papers, 7 papers with code
DSI++: Updating Transformer Memory with New Documents
In this work, we introduce DSI++, a continual learning challenge for DSI to incrementally index new documents while being able to answer queries related to both previously and newly indexed documents.
Transcending Scaling Laws with 0.1% Extra Compute
This paper proposes UL2R, a method that substantially improves existing language models and their scaling curves with a relatively tiny amount of extra compute.
Ranked #2 on
Cross-Lingual Question Answering
on TyDiQA-GoldP
Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?
There have been a lot of interest in the scaling properties of Transformer models.
ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning
Despite the recent success of multi-task learning and transfer learning for natural language processing (NLP), few works have systematically studied the effect of scaling up the number of tasks during pre-training.
Improving Compositional Generalization with Self-Training for Data-to-Text Generation
Data-to-text generation focuses on generating fluent natural language responses from structured meaning representations (MRs).
Scale Efficiently: Insights from Pretraining and Finetuning Transformers
The key findings of this paper are as follows: (1) we show that aside from only the model size, model shape matters for downstream fine-tuning, (2) scaling protocols operate differently at different compute regions, (3) widely adopted T5-base and T5-large sizes are Pareto-inefficient.
Scale Efficiently: Insights from Pre-training and Fine-tuning Transformers
The key findings of this paper are as follows: (1) we show that aside from only the model size, model shape matters for downstream fine-tuning, (2) scaling protocols operate differently at different compute regions, (3) widely adopted T5-base and T5-large sizes are Pareto-inefficient.
Long Range Arena : A Benchmark for Efficient Transformers
Transformers do not scale very well to long sequence lengths largely because of quadratic self-attention complexity.
Long Range Arena: A Benchmark for Efficient Transformers
In the recent months, a wide spectrum of efficient, fast Transformers have been proposed to tackle this problem, more often than not claiming superior or comparable model quality to vanilla Transformer models.
Ranked #19 on
Long-range modeling
on LRA
Bridging the Gap between Relevance Matching and Semantic Matching for Short Text Similarity Modeling
A core problem of information retrieval (IR) is relevance matching, which is to rank documents by relevance to a user{'}s query.
The OSU/Facebook Realizer for SRST 2019: Seq2Seq Inflection and Serialized Tree2Tree Linearization
We describe our exploratory system for the shallow surface realization task, which combines morphological inflection using character sequence-to-sequence models with a baseline linearizer that implements a tree-to-tree model using sequence-to-sequence models on serialized trees.
Incorporating Contextual and Syntactic Structures Improves Semantic Similarity Modeling
Semantic similarity modeling is central to many NLP problems such as natural language inference and question answering.
A Tree-to-Sequence Model for Neural NLG in Task-Oriented Dialog
Generating fluent natural language responses from structured semantic representations is a critical step in task-oriented conversational systems.
Constrained Decoding for Neural NLG from Compositional Representations in Task-Oriented Dialogue
Generating fluent natural language responses from structured semantic representations is a critical step in task-oriented conversational systems.
Lightweight and Efficient Neural Natural Language Processing with Quaternion Networks
Many state-of-the-art neural models for NLP are heavily parameterized and thus memory inefficient.
Simple and Effective Curriculum Pointer-Generator Networks for Reading Comprehension over Long Narratives
This paper tackles the problem of reading comprehension over long narratives where documents easily span over thousands of tokens.
Simple Attention-Based Representation Learning for Ranking Short Social Media Posts
This paper explores the problem of ranking short social media posts with respect to user queries using neural networks.
Multi-Perspective Relevance Matching with Hierarchical ConvNets for Social Media Search
To our best knowledge, this paper presents the first substantial work tackling search over social media posts using neural ranking models.
Exploring the Effectiveness of Convolutional Neural Networks for Answer Selection in End-to-End Question Answering
Most work on natural language question answering today focuses on answer selection: given a candidate list of sentences, determine which contains the answer.
Integrating Lexical and Temporal Signals in Neural Ranking Models for Searching Social Media Streams
To our knowledge, we are the first to integrate lexical and temporal signals in an end-to-end neural network architecture, in which existing neural ranking models are used to generate query-document similarity vectors that feed into a bidirectional LSTM layer for temporal modeling.
