Search Results for author:
Found 13 papers, 6 papers with code
Channel Permutations for N:M Sparsity
We introduce channel permutations as a method to maximize the accuracy of N:M sparse networks.
Learning both Weights and Connections for Efficient Neural Networks
On the ImageNet dataset, our method reduced the number of parameters of AlexNet by a factor of 9x, from 61 million to 6. 7 million, without incurring accuracy loss.
Efficient Sparse-Winograd Convolutional Neural Networks
First, we move the ReLU operation into the Winograd domain to increase the sparsity of the transformed activations.
Accelerating Sparse Deep Neural Networks
We present the design and behavior of Sparse Tensor Cores, which exploit a 2:4 (50%) sparsity pattern that leads to twice the math throughput of dense matrix units.
DSD: Dense-Sparse-Dense Training for Deep Neural Networks
We propose DSD, a dense-sparse-dense training flow, for regularizing deep neural networks and achieving better optimization performance.
Self-Supervised GAN Compression
Deep learning's success has led to larger and larger models to handle more and more complex tasks; trained models can contain millions of parameters.
Structurally Sparsified Backward Propagation for Faster Long Short-Term Memory Training
Further, we can enforce structured sparsity in the gate gradients to make the LSTM backward pass up to 45% faster than the state-of-the-art dense approach and 168% faster than the state-of-the-art sparsifying method on modern GPUs.
Sparse Persistent RNNs: Squeezing Large Recurrent Networks On-Chip
Recurrent Neural Networks (RNNs) are powerful tools for solving sequence-based problems, but their efficacy and execution time are dependent on the size of the network.
Exploring the Regularity of Sparse Structure in Convolutional Neural Networks
Since memory reference is more than two orders of magnitude more expensive than arithmetic operations, the regularity of sparse structure leads to more efficient hardware design.
Compressing DMA Engine: Leveraging Activation Sparsity for Training Deep Neural Networks
Popular deep learning frameworks require users to fine-tune their memory usage so that the training data of a deep neural network (DNN) fits within the GPU physical memory.
Learning both Weights and Connections for Efficient Neural Network
On the ImageNet dataset, our method reduced the number of parameters of AlexNet by a factor of 9×, from 61 million to 6. 7 million, without incurring accuracy loss.
Buddy Compression: Enabling Larger Memory for Deep Learning and HPC Workloads on GPUs
However, GPU device memory tends to be relatively small and the memory capacity can not be increased by the user.
Hardware Architecture
Self-Supervised Generative Adversarial Compression
Deep learning’s success has led to larger and larger models to handle more and more complex tasks; trained models often contain millions of parameters.
