Search Results for author:
Found 14 papers, 1 papers with code
FiD-Light: Efficient and Effective Retrieval-Augmented Text Generation
Retrieval-augmented generation models offer many benefits over standalone language models: besides a textual answer to a given query they provide provenance items retrieved from an updateable knowledge base.
Multi-Task Retrieval-Augmented Text Generation with Relevance Sampling
This paper studies multi-task training of retrieval-augmented generation models for knowledge-intensive tasks.
Transforming Sequence Tagging Into A Seq2Seq Task
Pretrained, large, generative language models (LMs) have had great success in a wide range of sequence tagging and structured prediction tasks.
WIT: Wikipedia-based Image Text Dataset for Multimodal Multilingual Machine Learning
First, WIT is the largest multimodal dataset by the number of image-text examples by 3x (at the time of writing).
Ranked #1 on
Image Retrieval
on WIT
DICT-MLM: Improved Multilingual Pre-Training using Bilingual Dictionaries
In particular, by requiring the model to predict the language-specific token, the MLM objective disincentivizes learning a language-agnostic representation -- which is a key goal of multilingual pre-training.
DiPair: Fast and Accurate Distillation for Trillion-Scale Text Matching and Pair Modeling
Pre-trained models like BERT (Devlin et al., 2018) have dominated NLP / IR applications such as single sentence classification, text pair classification, and question answering.
Sampled Softmax with Random Fourier Features
For the settings where a large number of classes are involved, a common method to speed up training is to sample a subset of classes and utilize an estimate of the loss gradient based on these classes, known as the sampled softmax method.
Distinct Sampling on Streaming Data with Near-Duplicates
In this paper we study how to perform distinct sampling in the streaming model where data contain near-duplicates.
Data Structures and Algorithms
Stochastic Negative Mining for Learning with Large Output Spaces
Furthermore, we identify a particularly intuitive class of loss functions in the aforementioned family and show that they are amenable to practical implementation in the large output space setting (i. e. computation is possible without evaluating scores of all labels) by developing a technique called Stochastic Negative Mining.
A Practical Algorithm for Distributed Clustering and Outlier Detection
We study the classic $k$-means/median clustering, which are fundamental problems in unsupervised learning, in the setting where data are partitioned across multiple sites, and where we are allowed to discard a small portion of the data by labeling them as outliers.
Tight Bounds for Collaborative PAC Learning via Multiplicative Weights
We study the collaborative PAC learning problem recently proposed in Blum et al.~\cite{BHPQ17}, in which we have $k$ players and they want to learn a target function collaboratively, such that the learned function approximates the target function well on all players' distributions simultaneously.
Adapting Kernel Representations Online Using Submodular Maximization
In this work, we develop an approximately submodular criterion for this setting, and an efficient online greedy submodular maximization algorithm for optimizing the criterion.
Adaptive Multiple-Arm Identification
We study the problem of selecting $K$ arms with the highest expected rewards in a stochastic $n$-armed bandit game.
Communication-Optimal Distributed Clustering
We would like the quality of the clustering in the distributed setting to match that in the centralized setting for which all the data resides on a single site.
