Search Results for author:
Found 31 papers, 9 papers with code
Towards Disentangling Relevance and Bias in Unbiased Learning to Rank
We give both theoretical analysis and empirical results to show the negative effects on relevance tower due to such a correlation.
Learning List-Level Domain-Invariant Representations for Ranking
Domain adaptation aims to transfer the knowledge acquired by models trained on (data-rich) source domains to (low-resource) target domains, for which a popular method is invariant representation learning.
Regression Compatible Listwise Objectives for Calibrated Ranking
In this paper, we propose a novel regression compatible ranking (RCR) approach to achieve a better trade-off.
Adaptive Combination of Proportionate Recursive Maximum Correntropy Criterion Algorithms and its Performance Analysis
The sparse adaptive algorithms under maximum correntropy criterion (MCC) have been developed and available for practical use due to their robustness against outliers (or impulsive noises).
The Devil in Linear Transformer
In this paper, we examine existing kernel-based linear transformers and identify two key issues that lead to such performance gaps: 1) unbounded gradients in the attention computation adversely impact the convergence of linear transformer models; 2) attention dilution which trivially distributes attention scores over long sequences while neglecting neighbouring structures.
Linear Video Transformer with Feature Fixation
Therefore, we propose a feature fixation module to reweight the feature importance of the query and key before computing linear attention.
RankT5: Fine-Tuning T5 for Text Ranking with Ranking Losses
Recently, substantial progress has been made in text ranking based on pretrained language models such as BERT.
Retrieval Augmentation for T5 Re-ranker using External Sources
Retrieval augmentation has shown promising improvements in different tasks.
A Validation Approach to Over-parameterized Matrix and Image Recovery
In this paper, we study the problem of recovering a low-rank matrix from a number of noisy random linear measurements.
Neural Architecture Search on Efficient Transformers and Beyond
To address this issue, we propose a new framework to find optimal architectures for efficient Transformers with the neural architecture search (NAS) technique.
Error Analysis of Tensor-Train Cross Approximation
Tensor train decomposition is widely used in machine learning and quantum physics due to its concise representation of high-dimensional tensors, overcoming the curse of dimensionality.
Vicinity Vision Transformer
Based on this observation, we present a Vicinity Attention that introduces a locality bias to vision transformers with linear complexity.
ED2LM: Encoder-Decoder to Language Model for Faster Document Re-ranking Inference
This results in significant inference time speedups since the decoder-only architecture only needs to learn to interpret static encoder embeddings during inference.
cosFormer: Rethinking Softmax in Attention
As one of its core components, the softmax attention helps to capture long-range dependencies yet prohibits its scale-up due to the quadratic space and time complexity to the sequence length.
Transformer Memory as a Differentiable Search Index
In this paper, we demonstrate that information retrieval can be accomplished with a single Transformer, in which all information about the corpus is encoded in the parameters of the model.
Rank4Class: A Ranking Formulation for Multiclass Classification
Multiclass classification (MCC) is a fundamental machine learning problem of classifying each instance into one of a predefined set of classes.
Improving Neural Ranking via Lossless Knowledge Distillation
We explore a novel perspective of knowledge distillation (KD) for learning to rank (LTR), and introduce Self-Distilled neural Rankers (SDR), where student rankers are parameterized identically to their teachers.
Rank4Class: Examining Multiclass Classification through the Lens of Learning to Rank
We further demonstrate that the most popular MCC architecture in deep learning can be mathematically formulated as a LTR pipeline equivalently, with a specific set of choices in terms of ranking model architecture and loss function.
Are Pretrained Convolutions Better than Pretrained Transformers?
In the context of language models, are convolutional models competitive to Transformers when pre-trained?
Charformer: Fast Character Transformers via Gradient-based Subword Tokenization
In this paper, we propose a new model inductive bias that learns a subword tokenization end-to-end as part of the model.
Ranked #3 on
Paraphrase Identification
on Quora Question Pairs
Are Pre-trained Convolutions Better than Pre-trained Transformers?
In the context of language models, are convolutional models competitive to Transformers when pre-trained?
OmniNet: Omnidirectional Representations from Transformers
In OmniNet, instead of maintaining a strictly horizontal receptive field, each token is allowed to attend to all tokens in the entire network.
Ranked #1 on
Machine Translation
on WMT2017 Russian-English
Neural Rankers are hitherto Outperformed by Gradient Boosted Decision Trees
We first validate this concern by showing that most recent neural LTR models are, by a large margin, inferior to the best publicly available Gradient Boosted Decision Trees (GBDT) in terms of their reported ranking accuracy on benchmark datasets.
DeepKeyGen: A Deep Learning-based Stream Cipher Generator for Medical Image Encryption and Decryption
In this paper, a novel deep learning-based key generation network (DeepKeyGen) is proposed as a stream cipher generator to generate the private key, which can then be used for encrypting and decrypting of medical images.
Diversify Question Generation with Continuous Content Selectors and Question Type Modeling
However, question generation is actually a one-to-many problem, as it is possible to raise questions with different focuses on contexts and various means of expression.
Do RNN and LSTM have Long Memory?
The LSTM network was proposed to overcome the difficulty in learning long-term dependence, and has made significant advancements in applications.
Non-Clicks Mean Irrelevant? Propensity Ratio Scoring As a Correction
Recent advances in unbiased learning to rank (LTR) count on Inverse Propensity Scoring (IPS) to eliminate bias in implicit feedback.
An Improvement of PAA on Trend-Based Approximation for Time Series
In this paper, we propose two new approaches for time series that utilize approximate trend feature information.
Multi-Task Learning for Email Search Ranking with Auxiliary Query Clustering
In this paper, we study how to obtain query type in an unsupervised fashion and how to incorporate this information into query-dependent ranking models.
An Online Learned Elementary Grouping Model for Multi-target Tracking
We introduce an online approach to learn possible elementary groups (groups that contain only two targets) for inferring high level context that can be used to improve multi-target tracking in a data-association based framework.
Efficient Online Bootstrapping for Large Scale Learning
Bootstrapping is a useful technique for estimating the uncertainty of a predictor, for example, confidence intervals for prediction.
