Model Compression
Leaderboards
Greatest papers with code
The State of Sparsity in Deep Neural Networks
• google-research/google-research
• 
We rigorously evaluate three state-of-the-art techniques for inducing sparsity in deep neural networks on two large-scale learning tasks: Transformer trained on WMT 2014 English-to-German, and ResNet-50 trained on ImageNet.
SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size
• pytorch/vision
• 
(2) Smaller DNNs require less bandwidth to export a new model from the cloud to an autonomous car.
Model compression via distillation and quantization
ICLR 2018
• NervanaSystems/distiller
•
Deep neural networks (DNNs) continue to make significant advances, solving tasks from image classification to translation or reinforcement learning.
AMC: AutoML for Model Compression and Acceleration on Mobile Devices
ECCV 2018
• NervanaSystems/distiller
•
Model compression is a critical technique to efficiently deploy neural network models on mobile devices which have limited computation resources and tight power budgets.
Global Sparse Momentum SGD for Pruning Very Deep Neural Networks
NeurIPS 2019
• ShawnDing1994/ACNet
•
Deep Neural Network (DNN) is powerful but computationally expensive and memory intensive, thus impeding its practical usage on resource-constrained front-end devices.
Contrastive Representation Distillation
We demonstrate that this objective ignores important structural knowledge of the teacher network.
Data-Free Knowledge Distillation for Deep Neural Networks
• huawei-noah/DAFL
•
Recent advances in model compression have provided procedures for compressing large neural networks to a fraction of their original size while retaining most if not all of their accuracy.
Discrimination-aware Channel Pruning for Deep Neural Networks
NeurIPS 2018
• SCUT-AILab/DCP
•
Channel pruning is one of the predominant approaches for deep model compression.
A Programmable Approach to Model Compression
• NVlabs/condensa
•
However, while the results are desirable, finding the best compression strategy for a given neural network, target platform, and optimization objective often requires extensive experimentation.
IMAGE CLASSIFICATION LANGUAGE MODELLING MODEL COMPRESSION QUANTIZATION
MicroExpNet: An Extremely Small and Fast Model For Expression Recognition From Face Images
On the other hand, KD is proved to be useful for model compression for the FER problem, and we discovered that its effects gets more and more significant with the decreasing model size.