(2) Smaller DNNs require less bandwidth to export a new model from the cloud to an autonomous car.
We rigorously evaluate three state-of-the-art techniques for inducing sparsity in deep neural networks on two large-scale learning tasks: Transformer trained on WMT 2014 English-to-German, and ResNet-50 trained on ImageNet.
Model compression is a critical technique to efficiently deploy neural network models on mobile devices which have limited computation resources and tight power budgets.
Channel pruning is one of the predominant approaches for deep model compression.
This paper is aimed at creating extremely small and fast convolutional neural networks (CNN) for the problem of facial expression recognition (FER) from frontal face images.
Making deep convolutional neural networks more accurate typically comes at the cost of increased computational and memory resources.
This work aims to learn structurally-sparse Long Short-Term Memory (LSTM) by reducing the sizes of basic structures within LSTM units, including input updates, gates, hidden states, cell states and outputs.
2) Cross-layer filter comparison is unachievable since the importance is defined locally within each layer.