Search Results for author:
Found 5 papers, 2 papers with code
Inducing and Exploiting Activation Sparsity for Fast Inference on Deep Neural Networks
In this paper, we present an in-depth analysis of methods for maximizing the sparsity of the activations in a trained neural network, and show that, when coupled with an efficient sparse-input convolution algorithm, we can leverage this sparsity for significant performance gains.
oBERTa: Improving Sparse Transfer Learning via improved initialization, distillation, and pruning regimes
In this paper, we introduce the range of oBERTa language models, an easy-to-use set of language models which allows Natural Language Processing (NLP) practitioners to obtain between 3. 8 and 24. 3 times faster models without expertise in model compression.
Sparse*BERT: Sparse Models Generalize To New tasks and Domains
Our experimentation shows that models that are pruned during pretraining using general domain masked language models can transfer to novel domains and tasks without extensive hyperparameter exploration or specialized approaches.
The Optimal BERT Surgeon: Scalable and Accurate Second-Order Pruning for Large Language Models
We perform an in-depth study of the accuracy-compression trade-off for unstructured weight pruning of BERT models.
How Well Do Sparse Imagenet Models Transfer?
Transfer learning is a classic paradigm by which models pretrained on large "upstream" datasets are adapted to yield good results on "downstream" specialized datasets.
