Search Results for author:
Found 17 papers, 6 papers with code
A Proximal Algorithm for Network Slimming
This time-consuming, three-step process is a result of using subgradient descent to train CNNs.
Feature Affinity Assisted Knowledge Distillation and Quantization of Deep Neural Networks on Label-Free Data
In this paper, we propose a feature affinity (FA) assisted knowledge distillation (KD) method to improve quantization-aware training of deep neural networks (DNN).
Searching Intrinsic Dimensions of Vision Transformers
It has been shown by many researchers that transformers perform as well as convolutional neural networks in many computer vision tasks.
Improving Network Slimming with Nonconvex Regularization
Network slimming with T$\ell_1$ regularization also outperforms the latest Bayesian modification of network slimming in compressing a CNN architecture in terms of memory storage while preserving its model accuracy after channel pruning.
RARTS: An Efficient First-Order Relaxed Architecture Search Method
Differentiable architecture search (DARTS) is an effective method for data-driven neural network design based on solving a bilevel optimization problem.
TRP: Trained Rank Pruning for Efficient Deep Neural Networks
The TRP trained network inherently has a low-rank structure, and is approximated with negligible performance loss, thus eliminating the fine-tuning process after low rank decomposition.
$\ell_0$ Regularized Structured Sparsity Convolutional Neural Networks
Deepening and widening convolutional neural networks (CNNs) significantly increases the number of trainable weight parameters by adding more convolutional layers and feature maps per layer, respectively.
Weakly-Supervised Degree of Eye-Closeness Estimation
To alleviate this issue, we propose a weakly-supervised method which utilizes the accurate annotation from the synthetic data set, to learn accurate degree of eye openness, and the weakly labeled (open or closed) real world eye data set to control the domain shift.
Trained Rank Pruning for Efficient Deep Neural Networks
To accelerate DNNs inference, low-rank approximation has been widely adopted because of its solid theoretical rationale and efficient implementations.
Understanding Straight-Through Estimator in Training Activation Quantized Neural Nets
We prove that if the STE is properly chosen, the expected coarse gradient correlates positively with the population gradient (not available for the training), and its negation is a descent direction for minimizing the population loss.
AutoShuffleNet: Learning Permutation Matrices via an Exact Lipschitz Continuous Penalty in Deep Convolutional Neural Networks
In addition, we found experimentally that the standard convex relaxation of permutation matrices into stochastic matrices leads to poor performance.
DAC: Data-free Automatic Acceleration of Convolutional Networks
A complex deep learning model with high accuracy runs slowly on resource-limited devices, while a light-weight model that runs much faster loses accuracy.
DNQ: Dynamic Network Quantization
Network quantization is an effective method for the deployment of neural networks on memory and energy constrained mobile devices.
Trained Rank Pruning for Efficient Deep Neural Networks
We propose Trained Rank Pruning (TRP), which iterates low rank approximation and training.
Blended Coarse Gradient Descent for Full Quantization of Deep Neural Networks
We introduce the notion of coarse gradient and propose the blended coarse gradient descent (BCGD) algorithm, for training fully quantized neural networks.
BinaryRelax: A Relaxation Approach For Training Deep Neural Networks With Quantized Weights
We propose BinaryRelax, a simple two-phase algorithm, for training deep neural networks with quantized weights.
Quantization and Training of Low Bit-Width Convolutional Neural Networks for Object Detection
We present LBW-Net, an efficient optimization based method for quantization and training of the low bit-width convolutional neural networks (CNNs).
