Search Results for author:
Found 15 papers, 9 papers with code
EXAQ: Exponent Aware Quantization For LLMs Acceleration
The combination of accelerating both $e^x$ and $\sum(e^x)$ results in a 36. 9% acceleration in the softmax operation.
Scaling FP8 training to trillion-token LLMs
We train, for the first time, large language models using FP8 precision on datasets up to 2 trillion tokens -- a 20-fold increase over previous limits.
Bimodal Distributed Binarized Neural Networks
The proposed method consists of a training scheme that we call Weight Distribution Mimicking (WDM), which efficiently imitates the full-precision network weight distribution to their binary counterpart.
Minimum Variance Unbiased N:M Sparsity for the Neural Gradients
We show that while minimization of the MSE works fine for pruning the weights and activations, it catastrophically fails for the neural gradients.
Accurate Neural Training with 4-bit Matrix Multiplications at Standard Formats
Reducing the computational footprint of the entire training process requires the quantization of the neural gradients, i. e., the loss gradients with respect to the outputs of intermediate neural layers.
Logarithmic Unbiased Quantization: Practical 4-bit Training in Deep Learning
Based on this, we suggest a logarithmic unbiased quantization (LUQ) method to quantize both the forward and backward phase to 4-bit, achieving state-of-the-art results in 4-bit training.
Accelerated Sparse Neural Training: A Provable and Efficient Method to Find N:M Transposable Masks
Finally, to solve the problem of switching between different structure constraints, we suggest a method to convert a pre-trained model with unstructured sparsity to an N:M fine-grained block sparsity model with little to no training.
Neural gradients are near-lognormal: improved quantized and sparse training
While training can mostly be accelerated by reducing the time needed to propagate neural gradients back throughout the model, most previous works focus on the quantization/pruning of weights and activations.
Colored Noise Injection for Training Adversarially Robust Neural Networks
Even though deep learning has shown unmatched performance on various tasks, neural networks have been shown to be vulnerable to small adversarial perturbations of the input that lead to significant performance degradation.
Robust Quantization: One Model to Rule Them All
Neural network quantization methods often involve simulating the quantization process during training, making the trained model highly dependent on the target bit-width and precise way quantization is performed.
Loss Aware Post-training Quantization
We show that with more aggressive quantization, the loss landscape becomes highly non-separable with steep curvature, making the selection of quantization parameters more challenging.
Smoothed Inference for Adversarially-Trained Models
In this work, we study the application of randomized smoothing as a way to improve performance on unperturbed data as well as to increase robustness to adversarial attacks.
CAT: Compression-Aware Training for bandwidth reduction
Our method trains the model to achieve low-entropy feature maps, which enables efficient compression at inference time using classical transform coding methods.
Feature Map Transform Coding for Energy-Efficient CNN Inference
We analyze the performance of our approach on a variety of CNN architectures and demonstrate that FPGA implementation of ResNet-18 with our approach results in a reduction of around 40% in the memory energy footprint, compared to quantized network, with negligible impact on accuracy.
Towards Learning of Filter-Level Heterogeneous Compression of Convolutional Neural Networks
While mainstream deep learning methods train the neural networks weights while keeping the network architecture fixed, the emerging neural architecture search (NAS) techniques make the latter also amenable to training.
