Search Results for author:
Found 8 papers, 0 papers with code
Mitigating Outlier Activations in Low-Precision Fine-Tuning of Language Models
The benefit of using integers for outlier values is that it enables us to use operator tiling to avoid performing 16-bit integer matrix multiplication to address this problem effectively.
Statistical Hardware Design With Multi-model Active Learning
For instance, in our performance prediction setting, the proposed method needs 65% fewer samples to create the model, and in the design space exploration setting, our proposed method can find the best parameter settings by exploring less than 50 samples.
On the Convergence of Stochastic Gradient Descent in Low-precision Number Formats
Deep learning models are dominating almost all artificial intelligence tasks such as vision, text, and speech processing.
EuclidNets: An Alternative Operation for Efficient Inference of Deep Learning Models
With the advent of deep learning application on edge devices, researchers actively try to optimize their deployments on low-power and restricted memory devices.
Towards Fine-tuning Pre-trained Language Models with Integer Forward and Backward Propagation
Previously researchers were focused on lower bit-width integer data types for the forward propagation of language models to save memory and computation.
Is Integer Arithmetic Enough for Deep Learning Training?
As such, quantization has attracted the attention of researchers in recent years.
Rethinking Pareto Frontier for Performance Evaluation of Deep Neural Networks
We propose to use a multi-dimensional Pareto frontier to re-define the efficiency measure of candidate deep learning models, where several variables such as training cost, inference latency, and accuracy play a relative role in defining a dominant model.
CNN2Gate: Toward Designing a General Framework for Implementation of Convolutional Neural Networks on FPGA
This is due to the lower power consumption and easy reconfigurability offered by these platforms.
