By distinguishing and taking on the different significance of neuron responses and connections during learning, the generated network, namely JPnet, optimizes the sparsity of activations and weights for improving execution efficiency.
We describe an end-to-end neural network weight compression approach that draws inspiration from recent latent-variable data compression methods.
In this work, we present a new defense against membership inference attacks that preserves the utility of the target machine learning models significantly better than prior defenses.
Our key idea is to introduce a priority term that identifies the importance of a layer; we can select unimportant layers according to the priority and erase them after the training.
The relationship between the input feature maps and 2D kernels is revealed in a theoretical framework, based on which a kernel sparsity and entropy (KSE) indicator is proposed to quantitate the feature map importance in a feature-agnostic manner to guide model compression.
The workflow of a traditional pruning consists of three sequential stages: pre-training the original model, selecting the pre-trained filters via ranking according to a manually designed criterion (e. g., the norm of filters), and learning the remained filters via fine-tuning.
On-board processing elements on UAVs are currently inadequate for training and inference of Deep Neural Networks.