Search Results for author:
Found 8 papers, 2 papers with code
Hadamard Domain Training with Integers for Class Incremental Quantized Learning
In this paper, we propose a technique that leverages inexpensive Hadamard transforms to enable low-precision training with only integer matrix multiplications.
Augmenting Hessians with Inter-Layer Dependencies for Mixed-Precision Post-Training Quantization
To address this challenge, we propose a mixed-precision post training quantization (PTQ) approach that assigns different numerical precisions to tensors in a network based on their specific needs, for a reduced memory footprint and improved latency while preserving model accuracy.
Mixed Precision Post Training Quantization of Neural Networks with Sensitivity Guided Search
In this paper, we propose a method to efficiently determine quantization configurations of different tensors in ML models using post-training mixed precision quantization.
AutoDistill: an End-to-End Framework to Explore and Distill Hardware-Efficient Language Models
By evaluating on SQuAD, a model found by AutoDistill achieves an 88. 4% F1 score with 22. 8M parameters, which reduces parameters by more than 62% while maintaining higher accuracy than DistillBERT, TinyBERT, and NAS-BERT.
GLaM: Efficient Scaling of Language Models with Mixture-of-Experts
Scaling language models with more data, compute and parameters has driven significant progress in natural language processing.
Ranked #10 on
Language Modelling
on LAMBADA
Exploring the limits of Concurrency in ML Training on Google TPUs
Recent results in language understanding using neural networks have required training hardware of unprecedentedscale, with thousands of chips cooperating on a single training run.
Exploiting Parallelism Opportunities with Deep Learning Frameworks
State-of-the-art machine learning frameworks support a wide variety of design features to enable a flexible machine learning programming interface and to ease the programmability burden on machine learning developers.
Benchmarking TPU, GPU, and CPU Platforms for Deep Learning
Training deep learning models is compute-intensive and there is an industry-wide trend towards hardware specialization to improve performance.
