Model Compression
State-of-the-art leaderboards
Greatest papers with code
SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
• pytorch/vision
• 
(2) Smaller DNNs require less bandwidth to export a new model from the cloud to an autonomous car.
The State of Sparsity in Deep Neural Networks
• google-research/google-research
We rigorously evaluate three state-of-the-art techniques for inducing sparsity in deep neural networks on two large-scale learning tasks: Transformer trained on WMT 2014 English-to-German, and ResNet-50 trained on ImageNet.
Model compression via distillation and quantization
ICLR 2018
• NervanaSystems/distiller
•
Deep neural networks (DNNs) continue to make significant advances, solving tasks from image classification to translation or reinforcement learning.
AMC: AutoML for Model Compression and Acceleration on Mobile Devices
ECCV 2018
• NervanaSystems/distiller
•
Model compression is a critical technique to efficiently deploy neural network models on mobile devices which have limited computation resources and tight power budgets.
Discrimination-aware Channel Pruning for Deep Neural Networks
NeurIPS 2018
• SCUT-AILab/DCP
• 
Channel pruning is one of the predominant approaches for deep model compression.
MicroExpNet: An Extremely Small and Fast Model For Expression Recognition From Frontal Face Images
This paper is aimed at creating extremely small and fast convolutional neural networks (CNN) for the problem of facial expression recognition (FER) from frontal face images.
Dynamic Channel Pruning: Feature Boosting and Suppression
Making deep convolutional neural networks more accurate typically comes at the cost of increased computational and memory resources.
Learning Intrinsic Sparse Structures within Long Short-Term Memory
ICLR 2018
• wenwei202/iss-rnns
•
This work aims to learn structurally-sparse Long Short-Term Memory (LSTM) by reducing the sizes of basic structures within LSTM units, including input updates, gates, hidden states, cell states and outputs.
Ternary Weight Networks
We introduce ternary weight networks (TWNs) - neural networks with weights constrained to +1, 0 and -1.
Learning Efficient Detector with Semi-supervised Adaptive Distillation
• Tangshitao/Semi-supervised-Adaptive-Distillation
ADL enlarges the distillation loss for hard-to-learn and hard-to-mimic samples and reduces distillation loss for the dominant easy samples, enabling distillation to work on the single-stage detector first time, even if the student and the teacher are identical.