About

Benchmarks

TREND DATASET BEST METHOD PAPER TITLE PAPER CODE COMPARE

Greatest papers with code

Improving Neural Network Quantization without Retraining using Outlier Channel Splitting

28 Jan 2019NervanaSystems/distiller

The majority of existing literature focuses on training quantized DNNs, while this work examines the less-studied topic of quantizing a floating-point model without (re)training.

LANGUAGE MODELLING NEURAL NETWORK COMPRESSION QUANTIZATION

Forward and Backward Information Retention for Accurate Binary Neural Networks

CVPR 2020 JDAI-CV/dabnn

Our empirical study indicates that the quantization brings information loss in both forward and backward propagation, which is the bottleneck of training accurate binary neural networks.

BINARIZATION NEURAL NETWORK COMPRESSION QUANTIZATION

Data-Free Learning of Student Networks

ICCV 2019 huawei-noah/Data-Efficient-Model-Compression

Learning portable neural networks is very essential for computer vision for the purpose that pre-trained heavy deep models can be well applied on edge devices such as mobile phones and micro sensors.

NEURAL NETWORK COMPRESSION

Learning Sparse Networks Using Targeted Dropout

31 May 2019for-ai/TD

Before computing the gradients for each weight update, targeted dropout stochastically selects a set of units or weights to be dropped using a simple self-reinforcing sparsity criterion and then computes the gradients for the remaining weights.

NETWORK PRUNING NEURAL NETWORK COMPRESSION

Neural Network Compression Framework for fast model inference

20 Feb 2020openvinotoolkit/nncf

In this work we present a new framework for neural networks compression with fine-tuning, which we called Neural Network Compression Framework (NNCF).

BINARIZATION NEURAL NETWORK COMPRESSION QUANTIZATION

torchdistill: A Modular, Configuration-Driven Framework for Knowledge Distillation

25 Nov 2020yoshitomo-matsubara/torchdistill

While knowledge distillation (transfer) has been attracting attentions from the research community, the recent development in the fields has heightened the need for reproducible studies and highly generalized frameworks to lower barriers to such high-quality, reproducible deep learning research.

IMAGE CLASSIFICATION INSTANCE SEGMENTATION KNOWLEDGE DISTILLATION NEURAL NETWORK COMPRESSION

Similarity-Preserving Knowledge Distillation

ICCV 2019 yoshitomo-matsubara/torchdistill

Knowledge distillation is a widely applicable technique for training a student neural network under the guidance of a trained teacher network.

KNOWLEDGE DISTILLATION NEURAL NETWORK COMPRESSION

Soft Weight-Sharing for Neural Network Compression

13 Feb 2017KarenUllrich/Tutorial_BayesianCompressionForDL

The success of deep learning in numerous application domains created the de- sire to run and train them on mobile devices.

NEURAL NETWORK COMPRESSION QUANTIZATION

ZeroQ: A Novel Zero Shot Quantization Framework

CVPR 2020 amirgholami/ZeroQ

Importantly, ZeroQ has a very low computational overhead, and it can finish the entire quantization process in less than 30s (0. 5\% of one epoch training time of ResNet50 on ImageNet).

NEURAL NETWORK COMPRESSION QUANTIZATION

A Closer Look at Structured Pruning for Neural Network Compression

10 Oct 2018BayesWatch/pytorch-prunes

Structured pruning is a popular method for compressing a neural network: given a large trained network, one alternates between removing channel connections and fine-tuning; reducing the overall width of the network.

NETWORK PRUNING NEURAL NETWORK COMPRESSION