Search Results for author:
Found 23 papers, 10 papers with code
Dual-Encoders for Extreme Multi-Label Classification
We propose decoupled softmax loss - a simple modification to the InfoNCE loss - that overcomes the limitations of existing contrastive losses.
EHI: End-to-end Learning of Hierarchical Index for Efficient Dense Retrieval
Such techniques use a two-stage process: (a) contrastive learning to train a dual encoder to embed both the query and documents and (b) approximate nearest neighbor search (ANNS) for finding similar documents for a given query.
MatFormer: Nested Transformer for Elastic Inference
Furthermore, we observe that smaller encoders extracted from a universal MatFormer-based ViT (MatViT) encoder preserve the metric-space structure for adaptive large-scale retrieval.
Bayesian regularization of empirical MDPs
Due to noise, the policy learnedfrom the estimated model is often far from the optimal policy of the underlying model.
Positive Unlabeled Contrastive Learning
We extend this paradigm to the classical positive unlabeled (PU) setting, where the task is to learn a binary classifier given only a few labeled positive samples, and (often) a large amount of unlabeled samples (which could be positive or negative).
Accelerating Primal-dual Methods for Regularized Markov Decision Processes
Entropy regularized Markov decision processes have been widely used in reinforcement learning.
Extreme Zero-Shot Learning for Extreme Text Classification
To learn the semantic embeddings of instances and labels with raw text, we propose to pre-train Transformer-based encoders with self-supervised contrastive losses.
Multi Label Text Classification
Multi-Label Text Classification
+2
DRONE: Data-aware Low-rank Compression for Large NLP Models
In addition to compressing standard models, out method can also be used on distilled BERT models to further improve compression rate.
Approximate Newton policy gradient algorithms
This paper proposes an approximate Newton method for the policy gradient algorithm with entropy regularization.
Enterprise-Scale Search: Accelerating Inference for Sparse Extreme Multi-Label Ranking Trees
In industrial applications, these models operate at extreme scales, where every bit of performance is critical.
Top-$k$ eXtreme Contextual Bandits with Arm Hierarchy
We show that our algorithm has a regret guarantee of $O(k\sqrt{(A-k+1)T \log (|\mathcal{F}|T)})$, where $A$ is the total number of arms and $\mathcal{F}$ is the class containing the regression function, while only requiring $\tilde{O}(A)$ computation per time step.
Computational Efficiency
Extreme Multi-Label Classification
+2
Voting based ensemble improves robustness of defensive models
Several previous attempts tackled this problem by ensembling the soft-label prediction and have been proved vulnerable based on the latest attack methods.
On the Benefits of Multiple Gossip Steps in Communication-Constrained Decentralized Optimization
In this paper, we show that, in such compressed decentralized optimization settings, there are benefits to having {\em multiple} gossip steps between subsequent gradient iterations, even when the cost of doing so is appropriately accounted for e. g. by means of reducing the precision of compressed information.
Extreme Multi-label Classification from Aggregated Labels
Current XMC approaches are not built for such multi-instance multi-label (MIML) training data, and MIML approaches do not scale to XMC sizes.
Learning to Encode Position for Transformer with Continuous Dynamical Model
The main reason is that position information among input units is not inherently encoded, i. e., the models are permutation equivalent; this problem justifies why all of the existing models are accompanied by a sinusoidal encoding/embedding layer at the input.
Ranked #5 on
Semantic Textual Similarity
on MRPC
CAT: Customized Adversarial Training for Improved Robustness
Adversarial training has become one of the most effective methods for improving robustness of neural networks.
Think Globally, Act Locally: A Deep Neural Network Approach to High-Dimensional Time Series Forecasting
Forecasting high-dimensional time series plays a crucial role in many applications such as demand forecasting and financial predictions.
Taming Pretrained Transformers for Extreme Multi-label Text Classification
However, naively applying deep transformer models to the XMC problem leads to sub-optimal performance due to the large output space and the label sparsity issue.
Extreme Multi-Label Classification
General Classification
+4
Online Embedding Compression for Text Classification using Low Rank Matrix Factorization
Empirically, we show that the proposed method can achieve 90% compression with minimal impact in accuracy for sentence classification tasks, and outperforms alternative methods like fixed-point quantization or offline word embedding compression.
Kernel Ridge Regression via Partitioning
In this paper, we investigate a divide and conquer approach to Kernel Ridge Regression (KRR).
Structured Sparse Regression via Greedy Hard-Thresholding
Several learning applications require solving high-dimensional regression problems where the relevant features belong to a small number of (overlapping) groups.
Coordinate Descent Methods for Symmetric Nonnegative Matrix Factorization
Given a symmetric nonnegative matrix $A$, symmetric nonnegative matrix factorization (symNMF) is the problem of finding a nonnegative matrix $H$, usually with much fewer columns than $A$, such that $A \approx HH^T$.
NOMAD: Non-locking, stOchastic Multi-machine algorithm for Asynchronous and Decentralized matrix completion
One of the key features of NOMAD is that the ownership of a variable is asynchronously transferred between processors in a decentralized fashion.
Distributed, Parallel, and Cluster Computing
