Knowledge distillation (KD) has proved to be an effective approach for deep neural network compression, which learns a compact network (student) by transferring the knowledge from a pre-trained, over-parameterized network (teacher).

We study low-rank parameterizations of weight matrices with embedded spectral properties in the Deep Learning context.

IMAGE CLASSIFICATION IMAGE GENERATION NEURAL NETWORK COMPRESSION

We propose a radically different approach that: (i) employs analog memories to maximize the capacity of each memory cell, and (ii) jointly optimizes model compression and physical storage to maximize memory utility.

We can compress a neural network while exactly preserving its underlying functionality with respect to a given input domain if some of its neurons are stable.

Over-parameterization of neural networks is a well known issue that comes along with their great performance.

The recommendation system (RS) plays an important role in the content recommendation and retrieval scenarios.

FEATURE SELECTION NEURAL NETWORK COMPRESSION RECOMMENDATION SYSTEMS

We add loss terms for training the student that measure the dissimilarity between student and teacher outputs of the auxiliary classifiers.

IMAGE CLASSIFICATION KNOWLEDGE DISTILLATION NEURAL NETWORK COMPRESSION

The compression of deep neural networks (DNNs) to reduce inference cost becomes increasingly important to meet realistic deployment requirements of various applications.

We introduce Dirichlet pruning, a novel post-processing technique to transform a large neural network model into a compressed one.

In this paper, we present a comprehensive review of existing literature on compressing DNN model that reduces both storage and computation requirements.

KNOWLEDGE DISTILLATION NETWORK PRUNING NEURAL NETWORK COMPRESSION