Search Results for author:
Found 10 papers, 1 papers with code
Mitigating Outlier Activations in Low-Precision Fine-Tuning of Language Models
The benefit of using integers for outlier values is that it enables us to use operator tiling to avoid performing 16-bit integer matrix multiplication to address this problem effectively.
Mathematical Challenges in Deep Learning
Deep models are dominating the artificial intelligence (AI) industry since the ImageNet challenge in 2012.
Statistical Hardware Design With Multi-model Active Learning
For instance, in our performance prediction setting, the proposed method needs 65% fewer samples to create the model, and in the design space exploration setting, our proposed method can find the best parameter settings by exploring less than 50 samples.
On the Convergence of Stochastic Gradient Descent in Low-precision Number Formats
Deep learning models are dominating almost all artificial intelligence tasks such as vision, text, and speech processing.
EuclidNets: An Alternative Operation for Efficient Inference of Deep Learning Models
With the advent of deep learning application on edge devices, researchers actively try to optimize their deployments on low-power and restricted memory devices.
Training Integer-Only Deep Recurrent Neural Networks
Recurrent neural networks (RNN) are the backbone of many text and speech applications.
Towards Fine-tuning Pre-trained Language Models with Integer Forward and Backward Propagation
Previously researchers were focused on lower bit-width integer data types for the forward propagation of language models to save memory and computation.
Is Integer Arithmetic Enough for Deep Learning Training?
As such, quantization has attracted the attention of researchers in recent years.
Accelerated Gradient Methods for Sparse Statistical Learning with Nonconvex Penalties
A recent proposal generalizes Nesterov's AG method to the nonconvex setting.
A Causal Direction Test for Heterogeneous Populations
Our simulation result show that the proposed adjustment significantly improves the performance of the causal direction test statistic for heterogeneous data.
