Search Results for author:
Found 15 papers, 4 papers with code
MagR: Weight Magnitude Reduction for Enhancing Post-Training Quantization
For example, we achieve a Wikitext2 perplexity of 5. 95 on the LLaMA2-70B model for per-channel INT2 weight quantization without incurring any inference overhead.
COMQ: A Backpropagation-Free Algorithm for Post-Training Quantization
Within a fixed layer, COMQ treats all the scaling factor(s) and bit-codes as the variables of the reconstruction error.
Feature Affinity Assisted Knowledge Distillation and Quantization of Deep Neural Networks on Label-Free Data
In this paper, we propose a feature affinity (FA) assisted knowledge distillation (KD) method to improve quantization-aware training of deep neural networks (DNN).
Recurrence of Optimum for Training Weight and Activation Quantized Networks
Deep neural networks (DNNs) are quantized for efficient inference on resource-constrained platforms.
Learning Quantized Neural Nets by Coarse Gradient Method for Non-linear Classification
In this paper, we propose a class of STEs with certain monotonicity, and consider their applications to the training of a two-linear-layer network with quantized activation functions for non-linear multi-category classification.
Global Convergence and Geometric Characterization of Slow to Fast Weight Evolution in Neural Network Training for Classifying Linearly Non-Separable Data
In this paper, we study the dynamics of gradient descent in learning neural networks for classification problems.
Understanding Straight-Through Estimator in Training Activation Quantized Neural Nets
We prove that if the STE is properly chosen, the expected coarse gradient correlates positively with the population gradient (not available for the training), and its negation is a descent direction for minimizing the population loss.
Non-ergodic Convergence Analysis of Heavy-Ball Algorithms
For objective functions satisfying a relaxed strongly convex condition, the linear convergence is established under weaker assumptions on the step size and inertial parameter than made in the existing literature.
Adversarial Defense via Data Dependent Activation Function and Total Variation Minimization
We improve the robustness of Deep Neural Net (DNN) to adversarial attacks by using an interpolating function as the output activation.
Blended Coarse Gradient Descent for Full Quantization of Deep Neural Networks
We introduce the notion of coarse gradient and propose the blended coarse gradient descent (BCGD) algorithm, for training fully quantized neural networks.
Laplacian Smoothing Gradient Descent
We propose a class of very simple modifications of gradient descent and stochastic gradient descent.
BinaryRelax: A Relaxation Approach For Training Deep Neural Networks With Quantized Weights
We propose BinaryRelax, a simple two-phase algorithm, for training deep neural networks with quantized weights.
Deep Learning for Real-Time Crime Forecasting and its Ternarization
In this work, we first present a proper representation of crime data.
Stochastic Backward Euler: An Implicit Gradient Descent Algorithm for $k$-means Clustering
In this paper, we propose an implicit gradient descent algorithm for the classic $k$-means problem.
Quantization and Training of Low Bit-Width Convolutional Neural Networks for Object Detection
We present LBW-Net, an efficient optimization based method for quantization and training of the low bit-width convolutional neural networks (CNNs).
