|TREND||DATASET||BEST METHOD||PAPER TITLE||PAPER||CODE||COMPARE|
Knowledge distillation (KD) has proved to be an effective approach for deep neural network compression, which learns a compact network (student) by transferring the knowledge from a pre-trained, over-parameterized network (teacher).
We propose a radically different approach that: (i) employs analog memories to maximize the capacity of each memory cell, and (ii) jointly optimizes model compression and physical storage to maximize memory utility.
We can compress a neural network while exactly preserving its underlying functionality with respect to a given input domain if some of its neurons are stable.
Over-parameterization of neural networks is a well known issue that comes along with their great performance.
The compression of deep neural networks (DNNs) to reduce inference cost becomes increasingly important to meet realistic deployment requirements of various applications.
In this paper, we present a comprehensive review of existing literature on compressing DNN model that reduces both storage and computation requirements.