Search Results for author:
Found 23 papers, 7 papers with code
Enhancing Computation Efficiency in Large Language Models through Weight and Activation Quantization
Large Language Models (LLMs) are proficient in natural language processing tasks, but their deployment is often restricted by extensive parameter sizes and computational demands.
Token-Scaled Logit Distillation for Ternary Weight Generative Language Models
Generative Language Models (GLMs) have shown impressive performance in tasks such as text generation, understanding, and reasoning.
SPADE: Sparse Pillar-based 3D Object Detection Accelerator for Autonomous Driving
3D object detection using point cloud (PC) data is essential for perception pipelines of autonomous driving, where efficient encoding is key to meeting stringent resource and latency requirements.
Teacher Intervention: Improving Convergence of Quantization Aware Training for Ultra-Low Precision Transformers
Pre-trained Transformer models such as BERT have shown great success in a wide range of applications, but at the cost of substantial increases in model complexity.
Exploring Attention Map Reuse for Efficient Transformer Neural Networks
In this paper, we provide a comprehensive study on attention map reuse focusing on its ability to accelerate inference.
Automatic Network Adaptation for Ultra-Low Uniform-Precision Quantization
Uniform-precision neural network quantization has gained popularity since it simplifies densely packed arithmetic unit for high computing capability.
Understanding and Improving Knowledge Distillation for Quantization-Aware Training of Large Transformer Encoders
In particular, KD has been employed in quantization-aware training (QAT) of Transformer encoders like BERT to improve the accuracy of the student model with the reduced-precision weight parameters.
Learning from distinctive candidates to optimize reduced-precision convolution program on tensor cores
In this method, we devise a search space that explores the thread tile and warp sizes to increase the data reuse despite a large matrix operand of reduced-precision MMA.
Thermal Face Detection for High-Speed AI Thermometer
However, the computation-demanding nature of DNNs, along with the time-consuming fusion of video and thermal camera frames, raises hurdles for the cost-effective deployment of such AI thermometer systems.
NN-LUT: Neural Approximation of Non-Linear Operations for Efficient Transformer Inference
Non-linear operations such as GELU, Layer normalization, and Softmax are essential yet costly building blocks of Transformer models.
TernGEMM: GEneral Matrix Multiply Library with Ternary Weights for Fast DNN Inference
We propose TernGEMM, a special GEMM library using SIMD instructions for Deep Neural Network (DNN) inference with ternary weights and activations under 8-bit.
Layer-wise Pruning of Transformer Attention Heads for Efficient Language Modeling
While Transformer-based models have shown impressive language modeling performance, the large computation cost is often prohibitive for practical use.
Understanding the Role of Self Attention for Efficient Speech Recognition
Self-attention (SA) is a critical component of Transformer neural networks that have succeeded in automatic speech recognition (ASR).
Automatic Speech Recognition
Automatic Speech Recognition (ASR)
+1
Robust Machine Learning Systems: Challenges, Current Trends, Perspectives, and the Road Ahead
Machine Learning (ML) techniques have been rapidly adopted by smart Cyber-Physical Systems (CPS) and Internet-of-Things (IoT) due to their powerful decision-making capabilities.
Uniform-Precision Neural Network Quantization via Neural Channel Expansion
Uniform-precision neural network quantization has gained popularity thanks to its simple arithmetic unit densely packed for high computing capability.
Stochastic Precision Ensemble: Self-Knowledge Distillation for Quantized Deep Neural Networks
In this study, we propose stochastic precision ensemble training for QDNNs (SPEQ).
Hybrid 8-bit Floating Point (HFP8) Training and Inference for Deep Neural Networks
Reducing the numerical precision of data and computation is extremely effective in accelerating deep learning training workloads.
The Sooner The Better: Investigating Structure of Early Winning Lottery Tickets
In the case of ResNet50 on ImageNet, this comes to the winning ticket of 75:02% Top-1 accuracy at 80% pruning rate in only 22% of the total epochs for iterative pruning.
Accumulation Bit-Width Scaling For Ultra-Low Precision Training Of Deep Networks
Observing that a bad choice for accumulation precision results in loss of information that manifests itself as a reduction in variance in an ensemble of partial sums, we derive a set of equations that relate this variance to the length of accumulation and the minimum number of bits needed for accumulation.
Training Deep Neural Networks with 8-bit Floating Point Numbers
The state-of-the-art hardware platforms for training Deep Neural Networks (DNNs) are moving from traditional single precision (32-bit) computations towards 16 bits of precision -- in large part due to the high energy efficiency and smaller bit storage associated with using reduced-precision representations.
Bridging the Accuracy Gap for 2-bit Quantized Neural Networks (QNN)
Deep learning algorithms achieve high classification accuracy at the expense of significant computation cost.
PACT: Parameterized Clipping Activation for Quantized Neural Networks
We show, for the first time, that both weights and activations can be quantized to 4-bits of precision while still achieving accuracy comparable to full precision networks across a range of popular models and datasets.
AdaComp : Adaptive Residual Gradient Compression for Data-Parallel Distributed Training
Highly distributed training of Deep Neural Networks (DNNs) on future compute platforms (offering 100 of TeraOps/s of computational capacity) is expected to be severely communication constrained.
