Search Results for author:
Found 31 papers, 13 papers with code
InterroGate: Learning to Share, Specialize, and Prune Representations for Multi-task Learning
In this work, we propose \textit{InterroGate}, a novel multi-task learning (MTL) architecture designed to mitigate task interference while optimizing inference computational efficiency.
Think Big, Generate Quick: LLM-to-SLM for Fast Autoregressive Decoding
We investigate the combination of encoder-decoder LLMs with both encoder-decoder and decoder-only SLMs from different model families and only require fine-tuning of the SLM.
GPTVQ: The Blessing of Dimensionality for LLM Quantization
In this work we show that the size versus accuracy trade-off of neural network quantization can be significantly improved by increasing the quantization dimensionality.
The LLM Surgeon
Experimentally, our method can prune rows and columns from a range of OPT models and Llamav2-7B by 20%-30%, with a negligible loss in performance, and achieve state-of-the-art results in unstructured and semi-structured pruning of large language models.
VeRA: Vector-based Random Matrix Adaptation
Low-rank adapation (LoRA) is a popular method that reduces the number of trainable parameters when finetuning large language models, but still faces acute storage challenges when scaling to even larger models or deploying numerous per-user or per-task adapted models.
Efficient Neural PDE-Solvers using Quantization Aware Training
In the past years, the application of neural networks as an alternative to classical numerical methods to solve Partial Differential Equations has emerged as a potential paradigm shift in this century-old mathematical field.
QBitOpt: Fast and Accurate Bitwidth Reallocation during Training
By combining fast-to-compute sensitivities with efficient solvers during QAT, QBitOpt can produce mixed-precision networks with high task performance guaranteed to satisfy strict resource constraints.
Pruning vs Quantization: Which is Better?
We provide an extensive comparison between the two techniques for compressing deep neural networks.
MSViT: Dynamic Mixed-Scale Tokenization for Vision Transformers
The input tokens to Vision Transformers carry little semantic meaning as they are defined as regular equal-sized patches of the input image, regardless of its content.
Revisiting Single-gated Mixtures of Experts
Mixture of Experts (MoE) are rising in popularity as a means to train extremely large-scale models, yet allowing for a reasonable computational cost at inference time.
FP8 versus INT8 for efficient deep learning inference
We theoretically show the difference between the INT and FP formats for neural networks and present a plethora of post-training quantization and quantization-aware-training results to show how this theory translates to practice.
A Practical Mixed Precision Algorithm for Post-Training Quantization
We experimentally validate our proposed method on several computer vision tasks, natural language processing tasks and many different networks, and show that we can find mixed precision networks that provide a better trade-off between accuracy and efficiency than their homogeneous bit-width equivalents.
FP8 Quantization: The Power of the Exponent
We detail the choices that can be made for the FP8 format, including the important choice of the number of bits for the mantissa and exponent, and show analytically in which settings these choices give better performance.
Simple and Efficient Architectures for Semantic Segmentation
Though the state-of-the architectures for semantic segmentation, such as HRNet, demonstrate impressive accuracy, the complexity arising from their salient design choices hinders a range of model acceleration tools, and further they make use of operations that are inefficient on current hardware.
Overcoming Oscillations in Quantization-Aware Training
These effects are particularly pronounced in low-bit ($\leq$ 4-bits) quantization of efficient networks with depth-wise separable layers, such as MobileNets and EfficientNets.
Cyclical Pruning for Sparse Neural Networks
Current methods for pruning neural network weights iteratively apply magnitude-based pruning on the model weights and re-train the resulting model to recover lost accuracy.
Neural Network Quantization with AI Model Efficiency Toolkit (AIMET)
chapter 4) and quantization-aware training (QAT, cf.
Understanding and Overcoming the Challenges of Efficient Transformer Quantization
Finally, we show that transformer weights and embeddings can be quantized to ultra-low bit-widths, leading to significant memory savings with a minimum accuracy loss.
A White Paper on Neural Network Quantization
Neural network quantization is one of the most effective ways of achieving these savings but the additional noise it induces can lead to accuracy degradation.
Distilling Optimal Neural Networks: Rapid Search in Diverse Spaces
Second, a rapid evolutionary search finds a set of pareto-optimal architectures for any scenario using the accuracy predictor and on-device measurements.
Differentiable Joint Pruning and Quantization for Hardware Efficiency
We present a differentiable joint pruning and quantization (DJPQ) scheme.
Bayesian Bits: Unifying Quantization and Pruning
We introduce Bayesian Bits, a practical method for joint mixed precision quantization and pruning through gradient based optimization.
Up or Down? Adaptive Rounding for Post-Training Quantization
In this paper, we propose AdaRound, a better weight-rounding mechanism for post-training quantization that adapts to the data and the task loss.
LSQ+: Improving low-bit quantization through learnable offsets and better initialization
To solve this problem, we propose LSQ+, a natural extension of LSQ, wherein we introduce a general asymmetric quantization scheme with trainable scale and offset parameters that can learn to accommodate the negative activations.
Ranked #18 on
Quantization
on ImageNet
Conditional Channel Gated Networks for Task-Aware Continual Learning
Therefore, we additionally introduce a task classifier that predicts the task label of each example, to deal with settings in which a task oracle is not available.
Ranked #3 on
Continual Learning
on ImageNet-50 (5 tasks)
Learned Threshold Pruning
This is in contrast to other methods that search for per-layer thresholds via a computationally intensive iterative pruning and fine-tuning process.
Gradient $\ell_1$ Regularization for Quantization Robustness
We analyze the effect of quantizing weights and activations of neural networks on their loss and derive a simple regularization scheme that improves robustness against post-training quantization.
Taxonomy and Evaluation of Structured Compression of Convolutional Neural Networks
The success of deep neural networks in many real-world applications is leading to new challenges in building more efficient architectures.
Batch-Shaping for Learning Conditional Channel Gated Networks
To achieve this, we introduce a new residual block architecture that gates convolutional channels in a fine-grained manner.
Data-Free Quantization Through Weight Equalization and Bias Correction
This improves quantization accuracy performance, and can be applied to many common computer vision architectures with a straight forward API call.
Relaxed Quantization for Discretized Neural Networks
Neural network quantization has become an important research area due to its great impact on deployment of large models on resource constrained devices.
