In this work, we carefully select five datasets, including two in-domain datasets and three out-of-domain datasets with different levels of domain shift, and study the generalization of a deep model in a zero-shot setting.
Automating information extraction from form-like documents at scale is a pressing need due to its potential impact on automating business workflows across many industries like financial services, insurance, and healthcare.
We further demonstrate that the dominant neural MCC architecture can be formulated as a neural ranking framework with a specific set of design choices.
We explore a novel perspective of knowledge distillation (KD) for learning to rank (LTR), and introduce Self-Distilled neural Rankers (SDR), where student rankers are parameterized identically to their teachers.
We further demonstrate that the most popular MCC architecture in deep learning can be mathematically formulated as a LTR pipeline equivalently, with a specific set of choices in terms of ranking model architecture and loss function.
To this end, we both explore two different vocabulary composition methods, as well as propose three sampling methods which help in efficient incremental training for BERT-like models.
When experiencing an information need, users want to engage with a domain expert, but often turn to an information retrieval system, such as a search engine, instead.
We investigate the privacy and utility implications of applying dx-privacy, a variant of Local Differential Privacy, to BERT fine-tuning in NLU applications.
First, WIT is the largest multimodal dataset by the number of image-text examples by 3x (at the time of writing).
Ranked #1 on Text-Image Retrieval on WIT
We first validate this concern by showing that most recent neural LTR models are, by a large margin, inferior to the best publicly available Gradient Boosted Decision Trees (GBDT) in terms of their reported ranking accuracy on benchmark datasets.
2 code implementations • 22 Oct 2020 • Nicholas Monath, Avinava Dubey, Guru Guruganesh, Manzil Zaheer, Amr Ahmed, Andrew McCallum, Gokhan Mergen, Marc Najork, Mert Terzihan, Bryon Tjanaka, YuAn Wang, Yuchen Wu
The applicability of agglomerative clustering, for inferring both hierarchical and flat clustering, is limited by its scalability.
Pre-trained models like BERT (Devlin et al., 2018) have dominated NLP / IR applications such as single sentence classification, text pair classification, and question answering.
A hybrid approach, which leverages both semantic (deep neural network-based) and lexical (keyword matching-based) retrieval models, is proposed.
We propose a novel approach using representation learning for tackling the problem of extracting structured information from form-like document images.
Furthermore, there is usually only a small amount of initial training data available when building machine-learned models to solve such problems.
In order to better capture sentence level semantic relations within a document, we pre-train the model with a novel masked sentence block language modeling task in addition to the masked word language modeling task used by BERT.
This paper describes a machine learning algorithm for document (re)ranking, in which queries and documents are firstly encoded using BERT , and on top of that a learning-to-rank (LTR) model constructed with TF-Ranking (TFR)  is applied to further optimize the ranking performance.
It thus motivates us to study how to leverage cross-document interactions for learning-to-rank in the deep learning framework.
We propose TensorFlow Ranking, the first open source library for solving large-scale ranking problems in a deep learning framework.
To overcome this limitation, we propose a new framework for multivariate scoring functions, in which the relevance score of a document is determined jointly by multiple documents in the list.