Search Results for author:
Found 10 papers, 1 papers with code
ELIAS: End-to-End Learning to Index and Search in Large Output Spaces
A popular approach for dealing with the large label space is to arrange the labels into a shallow tree-based index and then learn an ML model to efficiently search this index via beam search.
FINGER: Fast Inference for Graph-based Approximate Nearest Neighbor Search
Approximate K-Nearest Neighbor Search (AKNNS) has now become ubiquitous in modern applications, for example, as a fast search procedure with two tower deep learning models.
Overcoming Catastrophic Forgetting by Generative Regularization
In this paper, we propose a new method to overcome catastrophic forgetting by adding generative regularization to Bayesian inference framework.
MulCode: A Multiplicative Multi-way Model for Compressing Neural Language Model
For example, input embedding and Softmax matrices in IWSLT-2014 German-to-English data set account for more than 80{\%} of the total model parameters.
LEARNING TO LEARN WITH BETTER CONVERGENCE
We consider the learning to learn problem, where the goal is to leverage deeplearning models to automatically learn (iterative) optimization algorithms for training machine learning models.
Efficient Contextual Representation Learning With Continuous Outputs
Contextual representation models have achieved great success in improving various downstream natural language processing tasks.
Efficient Contextual Representation Learning Without Softmax Layer
Our framework reduces the time spent on the output layer to a negligible level, eliminates almost all the trainable parameters of the softmax layer and performs language modeling without truncating the vocabulary.
Learning to Screen for Fast Softmax Inference on Large Vocabulary Neural Networks
The algorithm achieves an order of magnitude faster inference than the original softmax layer for predicting top-$k$ words in various tasks such as beam search in machine translation or next words prediction.
GroupReduce: Block-Wise Low-Rank Approximation for Neural Language Model Shrinking
Model compression is essential for serving large deep neural nets on devices with limited resources or applications that require real-time responses.
A comparison of second-order methods for deep convolutional neural networks
Despite many second-order methods have been proposed to train neural networks, most of the results were done on smaller single layer fully connected networks, so we still cannot conclude whether it's useful in training deep convolutional networks.
